Data fluctuation attribution analysis method, system, and related devices

Abstract

The embodiment of the application discloses a data fluctuation attribution analysis method, system and related device, the method comprises the following steps: obtaining the current data of each sub-dimension under each dimension participating in statistics, obtaining historical statistical data different from the target statistical data collection time, calculating the data fluctuation measurement value corresponding to the current data of each sub-dimension under each sub-dimension, and determining the root cause dimension from the multiple dimensions according to the data fluctuation measurement value. Considering that the current data of a certain sub-dimension may be shared by other sub-dimensions under the same dimension, the application can not only quantitatively reflect the data changes of each sub-dimension level, but also feedback the influence of the current data of a certain sub-dimension on the current data of other sub-dimensions under the same dimension, so that the data mutual exclusion or data overlap scene between sub-dimensions can be finally analyzed, thereby accurately inferring which dimension in the upper layer is the root cause dimension causing the data fluctuation, and promoting the process of subsequent application.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of Internet, and in particular, to a data fluctuation attribution analysis method, system and related device. BACKGROUND

[0002] Nowadays, various industries often need to collect and analyze relevant data information to cope with process planning or job status in production and application processes. Therefore, in the face of constantly updated and changed data information, it is inevitable to effectively analyze the reasons for their changes.

[0003] Taking the daily active user (DAU) index as an example, in the existing scenario, the total DAU index of each day can be summarized and analyzed according to user registration age, user's current province, terminal device and other multi-dimensional data. However, if some users are in multiple provinces or use different types of terminal devices on the same day, the users participating in the statistics will belong to different dimension value (or sub-dimension) labels of a certain dimension, such as a user who is both a participant of a type a mobile phone sub-dimension and a participant of a type b mobile phone or even more sub-dimensions. The total DAU index of the day will have data overlap, making it easy to produce abnormal fluctuations in data volume and statistical results compared with the historical total DAU index.

[0004] However, the existing technology does not handle this data overlap well, such as directly ignoring cross-data from different dimension value levels, thereby blurring or missing the possible reasons for data fluctuations, and even incorrectly attributing the reasons to unrelated dimensions, thereby affecting subsequent data analysis results and job progress. SUMMARY

[0005] Embodiments of the present application provide a data fluctuation attribution analysis method, system and related device for accurately locating the root cause dimension causing data fluctuations.

[0006] The first aspect of the embodiments of the present application provides a data fluctuation attribution analysis method, comprising:

[0007] For the target statistical data summarized and analyzed by multi-dimensional data, the sub-dimension current data of each sub-dimension participating in the statistics is obtained, wherein the sub-dimension current data of at least part of the dimensions belongs to different sub-dimensions under the same dimension;

[0008] Obtain historical statistical data different from the target statistical data collection time;

[0009] For each dimension in the multi-dimension, according to the target statistical data and the historical statistical data, a data fluctuation metric value corresponding to the sub-dimension current data under each sub-dimension of the dimension is calculated respectively, the data fluctuation metric value being used to represent data change of the sub-dimension current data relative to the historical statistical data with participation of other sub-dimension current data in the same dimension;

[0010] According to the calculated data fluctuation metric values, a root cause dimension causing the target statistical data to have a fluctuation exceeding a threshold value relative to the historical statistical data is determined from the multi-dimension.

[0011] The second aspect of the embodiment of the present application provides a data fluctuation attribution analysis system, comprising: an acquisition unit, a calculation unit and a determination unit.

[0012] For target statistical data obtained through multi-dimension data summary analysis, the acquisition unit is configured to acquire sub-dimension current data participating in statistics under each sub-dimension of each dimension, wherein the sub-dimension current data of at least part of the dimensions belongs to different sub-dimensions in the same dimension.

[0013] The acquisition unit is further configured to acquire historical statistical data different from the target statistical data in time.

[0014] For each dimension in the multi-dimension, the calculation unit is configured to calculate, according to the target statistical data and the historical statistical data, a data fluctuation metric value corresponding to the sub-dimension current data under each sub-dimension of the dimension respectively, the data fluctuation metric value being used to represent data change of the sub-dimension current data relative to the historical statistical data with participation of other sub-dimension current data in the same dimension.

[0015] The determination unit is configured to determine, according to the calculated data fluctuation metric values, a root cause dimension causing the target statistical data to have a fluctuation exceeding a threshold value relative to the historical statistical data from the multi-dimension.

[0016] The third aspect of the embodiment of the present application provides a data fluctuation attribution analysis device, comprising:

[0017] A central processing unit, a memory and an input / output interface;

[0018] The memory is a transitory memory or a persistent memory;

[0019] The central processing unit is configured to communicate with the memory and execute instruction operation in the memory to perform the method described in the first aspect of the embodiment of the present application or any specific implementation manner of the first aspect.

[0020] The fourth aspect of the embodiment of the present application provides a computer readable storage medium comprising instructions which, when executed on a computer, cause the computer to perform the method described in the first aspect of the embodiment of the present application or any of the specific implementation manners of the first aspect.

[0021] The fifth aspect of the embodiment of the present application provides a computer program product comprising instructions which, when executed on a computer, cause the computer to perform the method described in the first aspect of the embodiment of the present application or any of the specific implementation manners of the first aspect.

[0022] From the above technical solutions, the embodiment of the present application has the following advantages:

[0023] Considering that the current data of a certain sub-dimension may be shared by other sub-dimensions under the same dimension, calculating the data fluctuation metric values corresponding to the current data of the sub-dimensions can not only quantitatively reflect the data changes at the level of each sub-dimension, but also feedback the influence of the current data of other sub-dimensions under the same dimension on the current data of a certain sub-dimension, so that the data mutual exclusion or data overlap scenario between sub-dimensions can be analyzed, and the root dimension causing the data fluctuation can be accurately inferred, thereby promoting the progress of subsequent applications. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art according to these drawings.

[0025] Figure 1 An application environment diagram of the embodiment of the present application;

[0026] Figure 2 A flowchart of the data fluctuation attribution analysis method of the embodiment of the present application;

[0027] Figure 3 Another flowchart of the data fluctuation attribution analysis method of the embodiment of the present application;

[0028] Figure 4 Another flowchart of the data fluctuation attribution analysis method of the embodiment of the present application;

[0029] Figure 5 Another flowchart of the data fluctuation attribution analysis method of the embodiment of the present application;

[0030] Figure 6 A structural diagram of the data fluctuation attribution analysis system of the embodiment of the present application;

[0031] Figure 7 A structural schematic diagram of an embodiment data fluctuation attribution analysis device of the present application. DETAILED DESCRIPTION

[0032] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings, and the described embodiments should not be regarded as limiting the present application. All other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0033] In the following description, the terms “one specific embodiment” or “one embodiment” and the like refer to a subset of all possible embodiments, but it can be understood that “one specific embodiment” or “one embodiment” can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict. In the following description, the term “plurality” refers to at least two.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0035] For the convenience of understanding and description, before the embodiments of the present application are further described in detail, a brief description of the conventional scheme will be given:

[0036] 1. The Adtributor attribution algorithm cannot well solve the case of data overlap between sub-dimensions. Taking the DAU index as an example, a user can appear in two or more sub-dimensions (or dimension values) at the same time, i.e., there are multiple sources, and the Adtributor algorithm cannot well perform source investigation and further data processing.

[0037] 2. The HotSpot and Squeeze attribution algorithms assume that the change rate of the root cause dimension is proportional to the change rate of its sub-dimension, but in most real scenarios, this assumption is too arbitrary, making the reliability of attribution analysis cannot be guaranteed.

[0038] 3. The iDice attribution algorithm uses a brute-force pruning method to reduce the search space, such as directly removing sub-dimension nodes with a change less than a certain threshold, which often directly affects the analysis of the upper dimension, thereby missing possible root cause dimensions.

[0039] Therefore, the present application provides a data fluctuation attribution analysis method to effectively solve the technical problem that the conventional scheme is not accurate enough in positioning the root cause dimension causing data fluctuation.

[0040] Please refer toFigure 1 , Figure 1 A schematic diagram illustrating an application environment applicable to embodiments of this application is shown. The data fluctuation attribution analysis method provided in embodiments of this application can be applied to, for example... Figure 1 The interactive system 100 shown includes a terminal device 101 and a server 102, which are communicatively connected to the terminal device 101. The server 102 can be a traditional server or a cloud server; no specific limitation is made here.

[0041] The terminal device 101 can be any electronic device with a display screen that supports data input, including but not limited to smartphones, tablets, laptops, desktop computers, and wearable electronic devices. Specifically, data input can be based on the voice module on the terminal device 101 for voice input, the character input module for character input, the image input module for image input, etc., or it can be based on the gesture recognition module installed on the terminal device 101, enabling users to perform interactive methods such as gesture input.

[0042] The terminal device 101 can be equipped with a client application product. Users can communicate with the server 102 based on the client application (such as an APP, WeChat mini-program, etc.). Specifically, the server 102 is equipped with a corresponding server application. Users can register a user account on the server 102 based on the client application and communicate with the server 102 based on the user account. For example, users can log in to their user account on the client application and input information through the client application, such as text information, voice information, or image information. After receiving the user input information, the client application can send the information to the server 102, so that the server 102 can receive the information, process it, and store it, such as to summarize and analyze data indicators such as the number of users of a certain client application. The server 102 can also receive the information and return a corresponding output information to the terminal device 101 based on the information.

[0043] The above application environment is merely an example for ease of understanding. It is understood that the embodiments of this application are not limited to the above application environment.

[0044] The embodiments of this application will be described in further detail below.

[0045] Please see Figure 2 The first aspect of this application provides an embodiment of a data fluctuation attribution analysis method, which includes:

[0046] 201. Obtain the current data of each sub-dimension participating in the statistics under each dimension.

[0047] Since the data constituting the target statistical data is essentially derived from the current data of each sub-dimension, in order to analyze the target statistical data, the current data of each sub-dimension participating in the statistics of each dimension needs to be obtained, wherein at least part of the current data of the sub-dimensions belongs to different sub-dimensions under the same dimension. Taking the DAU index as an example, the province dimension includes multiple sub-dimensions such as Hebei Province and Henan Province, that is, there are multiple specific dimension values under this dimension, and the user data information of a user recorded under the Hebei Province sub-dimension will also be recorded as a user participating in statistics under another sub-dimension (such as Henan Province) because the user moved across provinces on the same day, so the current data of the two provinces will overlap due to the presence of the user, such as adding one to the user count of the target statistical data on the same day.

[0048] 202. Obtain historical statistical data different from the target statistical data collection time.

[0049] In order to accurately determine the reason why the target statistical data has a threshold value fluctuation relative to the historical statistical data, that is, which dimension is the root cause, reference data for comparing the target statistical data is needed, and the reference data should be historical statistical data different from the target statistical data collection time, so as to help trace back to the root cause dimension that causes the target statistical data to have a threshold value fluctuation.

[0050] 203. Calculate the data fluctuation metric value corresponding to the current data of each sub-dimension of each dimension.

[0051] According to the target statistical data and the historical statistical data, the data fluctuation metric value corresponding to the current data of each sub-dimension of each dimension is calculated, and the data fluctuation metric value is used to represent the data change of the current data of the sub-dimension relative to the historical statistical data with the participation of other current data of the sub-dimension in the same dimension. Since a dimension can include multiple sub-dimensions, it can be understood that the data fluctuation metric value can also represent the fluctuation of data at the same dimension level before and after different collection times.

[0052] 204. According to the calculated data fluctuation metric values, determine the root cause dimension from the multiple dimensions.

[0053] Since the data fluctuation metric value can represent the fluctuation of data at the dimension level before and after different collection times, according to the calculated data fluctuation metric values, the root cause dimension that causes the target statistical data to have a threshold value fluctuation relative to the historical statistical data can be determined from the multiple dimensions.

[0054] It should be noted that the execution order of steps 201 and 202 is not limited.

[0055] In summary, in the process of calculating the data fluctuation metric value corresponding to each sub-dimension current data, the possibility that the current data of a certain sub-dimension is shared by other sub-dimensions under the same dimension is considered. Therefore, through the data fluctuation metric value, the data changes at each sub-dimension level can be quantified, and at the same time, the influence of the current data of a certain sub-dimension on the current data of other sub-dimensions under the same dimension can be fed back, so that the data mutual exclusion or data overlap between sub-dimensions can be analyzed, and the root cause dimension causing the data fluctuation can be accurately inferred to promote the subsequent application process, such as learning the popularity of a certain client application product among the public.

[0056] On the basis of the above example, some specific possible implementation examples will be provided below. In actual application, the implementation contents of these examples can be combined as needed according to the corresponding functional principles and application logic.

[0057] Please refer to Figure 3 The present application provides another embodiment of a data fluctuation attribution analysis method, which comprises:

[0058] 301. Obtain the sub-dimension current data of each sub-dimension under each dimension participating in statistics.

[0059] Still taking the DAU index as an example to illustrate the statistical data and its sub-dimension data of the present application, the multi-dimension DAU index target statistical data can cover not only the province dimension data but also the user terminal dimension data, and correspondingly, the user terminal dimension includes at least a plurality of sub-dimensions such as a type of mobile phone and b type of mobile phone. Similarly, for a user recorded in the a type of mobile phone sub-dimension, because he also used the b type of mobile phone to perform a related operation such as logging in to the same client application program on the same day, he is also recorded as a user participating in statistics under the b type of mobile phone sub-dimension. Therefore, the current data of the two mobile phone models will be cross-recorded due to the user, such as that the total DAU of the user terminal dimension on the same day is actually increased by one in counting, and even other related user information is repeatedly recorded and counted, resulting in disorder of the target statistical data. The total DAU can be understood as a data index at a certain dimension level. If such similar situations occur in a large number of users, the target statistical data obtained by the final aggregation analysis will be abnormally fluctuated beyond the threshold compared with the historical statistical data as a reference.

[0060] 302. Obtain historical statistical data different from the target statistical data collection time.

[0061] Specifically, the process of obtaining historical statistical data in the present application includes obtaining the dimension data and the lower-level sub-dimension data constituting the historical statistical data.

[0062] 303、Calculate the data fluctuation metric value corresponding to the sub-dimension current data of each sub-dimension of each dimension respectively.

[0063] As a possible implementation, if the data fluctuation metric value includes the probability distribution similarity corresponding to the sub-dimension current data and the index weight corresponding to the sub-dimension current data, step 303 can specifically include steps 3031 to 3035:

[0064] 3031、For each sub-dimension of the dimension, count the statistical object to which the sub-dimension current data of the sub-dimension points, and the sub-dimension experience quantity of the statistical object due to appearing in different sub-dimensions of the same dimension.

[0065] For example, for the jth sub-dimension of the ith dimension (which can be represented as sub-dimension or dimension value ij), the sub-dimension current data can specifically be the data index corresponding to the sub-dimension ij, such as the DAU index of Hebei Province on Thursday, and the statistical object to which the sub-dimension ij points can specifically be an individual user u with a registration number of 0123, which can be referred to as a data entity; correspondingly, N ij is the number of data entities under the dimension value ij, such as the number of users in the DAU index under the dimension value ij; the sub-dimension experience quantity of the statistical object u due to appearing in different sub-dimensions of the same dimension can be represented as w u , in other words, w u is the number of different dimension values in which the user u appears, for example, the user u appears in 3 dimension values, then w u = 3 to indicate that the 3 dimension values share the user u.

[0066] 3032、Calculate the prior probability of the sub-dimension experience quantity relative to the historical statistical data, and the posterior probability of the sub-dimension experience quantity relative to the target statistical data, the prior probability and the posterior probability being used to represent the probability distribution of the statistical object in the statistical data at different collection times.

[0067] On the basis of step 3031, the prior probability p ij of the sub-dimension experience quantity relative to the historical statistical data can be calculated according to the following formula:

[0068]

[0069] wherein F is the data corresponding to the ith dimension in the historical statistical data, such as the total DAU index value of the whole country on the last Thursday, which can be specifically understood as the total number of users from the whole country on the last Thursday;

[0070] Similarly, the posterior probability q ij of the sub-dimension experience quantity relative to the target statistical data can be calculated according to the following formula:

[0071]

[0072] M represents the data corresponding to the i-th dimension in the target statistical data, such as the total DAU index value of the whole country on this Thursday, which can be understood as the total number of users from the whole country on this Thursday.

[0073] 3033、Calculate the probability distribution similarity between the prior probability and the posterior probability.

[0074] For example, the probability distribution similarity is the Jensen-Shannon divergence, and the probability distribution similarity js divergence (S ij ) between the prior probability and the posterior probability is calculated according to the following formula:

[0075]

[0076] The greater the probability distribution similarity, the greater the data fluctuation amplitude caused by the data corresponding to the sub-dimension ij at different collection times.

[0077] From the above description and formula, it can be seen that the finally calculated probability distribution similarity can be used to represent the data difference degree of the target statistical data under the sub-dimension ij analyzed by the multi-dimension data summary. In this way, the probability distribution similarity (which can be simply referred to as the abnormality degree) under each dimension is calculated in turn, and the data difference degree caused by the target statistical data under different dimensions compared with the reference data can be fed back, and the root cause dimension causing the data abnormal fluctuation can be traced back.

[0078] Of course, other probability distribution similarities can also be used to represent the abnormality degree between the prior probability and the posterior probability, such as the Kullback-Leibler divergence.

[0079] Steps 3031 to 3033 are the specific implementation process of calculating the probability distribution similarity, and the specific implementation process of calculating the index weight will be described in detail below.

[0080] 3034、Replace the sub-dimension current data corresponding to the sub-dimension with the corresponding sub-dimension historical data from the historical statistical data under the same sub-dimension.

[0081] For example, for the j-th dimension value of the i-th dimension, such as the DAU index corresponding to Hebei Province, only replace the DAU value (which can be understood as the number of users) 100 of Hebei Province on this Thursday with the reference value 90 of the number of users of Hebei Province last Thursday, and the current data of other sub-dimensions under the province dimension except Hebei Province remains unchanged, that is, still the DAU value on this Thursday.

[0082] 3035、According to the replaced sub-dimension historical data, calculate the index weight of the change amount of the sub-dimension current data relative to the previous and next two statistical data.

[0083] On the basis of the above step 3034, specifically, according to the obtained sub-dimension historical data (such as the number of Hebei province users on last Thursday 90) and other sub-dimension current data under the same dimension, the dimension index value corresponding to the data of the certain dimension value ij is obtained after the reference value is replaced, such as the total number of users M of the province dimension at this time ij , and the index weight C of the sub-dimension current data relative to the change amount of the previous and next two statistical data is calculated according to the following formula ij

[0084]

[0085] Wherein, M and F are the same as the above-mentioned calculation of probability distribution similarity.

[0086] It can be seen that the index weight C ij The role is to assume that the current data of a certain sub-dimension is the same as the historical reference data of the same sub-dimension, so as to quantitatively represent the influence degree and contribution degree of the current data of the sub-dimension on the upper dimension data, such as quantifying the influence of the current Hebei province data on the national data, so the index weight of the present application can be called contribution degree. Of course, the index weight in the present application can be calculated in a form different from the above formula C ij , as long as it can also represent the data contribution degree, which is not limited here.

[0087] On the other hand, it can be understood that the calculation formula in steps 3031 to 3035 is not only applicable to the case where there is data overlap between the dimension values as above, but also applicable to the case where the data between the dimension values is mutually exclusive, which can be manifested as the current data of each sub-dimension under the same dimension is only attributed to one of the sub-dimensions, such as for each preset age interval (i.e. sub-dimension) under the age dimension, the user of 18 years old will only be attributed to or counted in the age interval [15, 20]. Of course, these formulas are also applicable to other counting type indicators other than DAU indicator, or ratio type derived indicators such as success rate.

[0088] It should be noted that the execution order of each step of calculating the probability distribution similarity and each step of calculating the index weight is not limited, i.e. any step in steps 3031 to 3033 does not require to be executed before or after any step in steps 3034 to 3035, as long as the required probability distribution similarity and index weight data fluctuation measurement value can be obtained eventually.

[0089] ​In some specific examples, the advantages of simultaneously using the probability distribution similarity and the index weight at least lie in helping to comprehensively consider the abnormality and contribution performance of the data, and thus helping to promote the real root cause dimension, so as to avoid a poor attribution result caused by only one of the two measurements. Taking the DAU index of the province dimension as an example, the abnormality of the current data corresponding to a certain sub-dimension may be large enough compared with the abnormality of the current data corresponding to other sub-dimensions in the same dimension, but the former mainly corresponds or collects the user data from a remote province on the same day, and the contribution of the user data from the remote province to the total user data (i.e. the upper dimension level) is small. Therefore, if only the abnormality is used as the basis, the dimension with large abnormality but small contribution is easily mistaken as the root cause dimension. Similarly, the contribution of a certain province on the same day may be large enough compared with the contribution of other provinces, but the abnormality of the former is small. At this time, if only the contribution is used as the basis, the dimension with large contribution but small abnormality is also easily mistaken as the root cause dimension, thereby affecting the timeliness and accuracy of the data fluctuation attribution process.

[0090] 304. Determine the root cause dimension from the multiple dimensions according to the calculated data fluctuation measurement values.

[0091] As a possible implementation, step 304 can specifically include:

[0092] For each dimension, it is determined whether there is a target number (such as K) of sub-dimensions in the dimension, each of which has a data fluctuation measurement value greater than the corresponding data fluctuation threshold value, and the data fluctuation threshold value includes a probability similarity threshold value and a weight threshold value. If there is, the dimension is determined as the root cause dimension. Of course, in this process, if the analyzed dimension is not the root cause dimension that meets the above condition (there are K sub-dimensions in the current dimension, each of which has a data fluctuation measurement value greater than the corresponding data fluctuation threshold value), the remaining dimensions are analyzed in the same way until the root cause dimension that meets the condition is found.

[0093] For example, Figure 4As shown, for example, the data fluctuation threshold used to compare the similarity of probability distributions is denoted as the anomaly threshold TS, and the data fluctuation threshold used to compare the index weights is denoted as the weight threshold TC. A multi-dimensional list (or a list of dimensions to be analyzed) associated with the target statistical data is compiled. For a given dimension, if there are K sub-dimensions under that dimension with anomalies greater than TS and contributions greater than TC, these K sub-dimensions can be considered as a combination of sub-dimensions corresponding to that dimension. Of course, these K sub-dimensions can specifically be the top K sub-dimensions sorted in descending order by a certain data fluctuation metric (such as anomaly). If such a combination exists under that dimension, then that dimension is added to the candidate root cause list to indicate that it is a root cause dimension; conversely, if no such combination exists under that dimension, the next dimension is analyzed in the same way until a dimension that can be used as a root cause dimension is found. Of course, the conditions for determining whether a dimension is a root cause dimension can also be set according to actual needs. For example, whether there are K sub-dimensions under a certain dimension whose weighted fusion value of anomaly and contribution reaches a certain threshold.

[0094] 305. Determine the final root cause dimension from multiple root cause dimensions.

[0095] If there are multiple root cause dimensions determined in step 304, the final root cause dimension can be determined in the following ways.

[0096] One approach is to sort the data fluctuation metrics corresponding to each root cause dimension in descending order, such as sorting each root cause dimension in descending order by anomaly, and selecting a preset number (such as the first) of the root cause dimensions as the final root cause dimensions.

[0097] Method Two: Merge the various data fluctuation metrics corresponding to each root cause dimension to obtain the result fusion value for each root cause dimension; select a preset number of root cause dimensions whose result fusion values ​​exceed the evaluation threshold as the final root cause dimensions. Alternatively, similar to Method One, the result fusion values ​​can be sorted in descending order, and a preset number of root cause dimensions with the highest sorted values ​​can be selected as the final root cause dimensions.

[0098] For example, taking the current data fluctuation metrics that only include anomaly and contribution as an example, the result fusion value of a certain root cause dimension = anomaly of that root cause dimension × anomaly weight + contribution of that root cause dimension × contribution weight. Of course, depending on actual needs, the determined multiple root cause dimensions can be directly reported to the required party, instead of inferring the final root cause dimension as in step 305 before reporting.

[0099] The operations described in steps 301 to 304 above are similar to those described in steps 201 to 204, and will not be repeated here.

[0100] In summary, when the statistical data such as the DAU index is abnormal, the abnormality degree and contribution degree of the data index under each dimension are quantified reasonably, the root cause dimension causing the abnormal fluctuation of the data is quickly located, the influence of the abnormal data is eliminated, and the subsequent work process is promoted. The abnormality degree and contribution degree formula of the present application considers both the data exclusion scenario between sub-dimensions and the data overlap scenario, so the present application can quickly troubleshoot the root cause of data changes in multiple scenarios.

[0101] Based on the embodiments shown in the above Figure 2 or Figure 3 In order to further improve the timeliness of locating the root cause dimension and speed up the data analysis process, before step 201 or 301 is executed, the method of the present application can further include querying and collecting target statistical data through the data scheduling platform.

[0102] Specifically, as shown in Figure 5 If the data fluctuation of the target statistical data relative to the historical statistical data exceeds the preset fluctuation range, the target statistical data is taken as the statistical data triggering the timing attribution analysis, and before the target statistical data starts the timing attribution analysis, whether the target statistical data is collected is queried through the data scheduling platform; if not, the data is continuously collected from the data source through the data scheduling platform until the collection is completed.

[0103] In actual application, the statistical data triggering the timing attribution analysis can specifically refer to a type of statistical data whose data fluctuation amplitude value exceeds the preset fluctuation range every period, such as the DAU index described above, therefore, at this time, it is necessary to rely on the data scheduling platform to periodically check whether this type of data to be analyzed is collected, so as to periodically start the intelligent attribution analysis step of Figure 2 or Figure 3 The example, and then generate the corresponding root cause dimension report to the virtual robot (such as an application product used to interact with the user), so that the user can finally know what the root cause dimension causing the abnormal fluctuation of the data is.

[0104] Of course, for a type of statistical data that does not exceed the preset fluctuation range every period, such as the ratio type derived index such as the successful login rate which is almost unchanged every day, because the data of this type of statistical data does not change much every period, it is not necessary to go through the timing attribution branch described above, so that the intelligent attribution analysis step is frequently carried out, but instead, the abnormality trigger attribution branch is used, so that the statistical data with little change is only developed to the intelligent attribution analysis step of Figure 2 or Figure 3 The example when the monitored change reaches a certain change amplitude threshold, in order to avoid resource occupation.

[0105] In the above Figures 2 to 5On the basis of any of the examples of the figures, to further improve the positioning accuracy of the root cause dimension, before performing step 201 or 301, the method of the application can further include the following steps:

[0106] For each dimension in the multi-dimension, if the sub-dimensions under the dimension satisfy the mutual exclusion relationship, the dimension is regarded as a dimension containing only mutually exclusive sub-dimensions, and the data fluctuation measure value includes the data difference value corresponding to the current data of the sub-dimension. The mutual exclusion relationship means that the current data of each sub-dimension under the same dimension is only attributed to one of the sub-dimensions. For example, for each preset age interval (i.e. sub-dimension) under the age dimension, a user of 18 years old is only attributed to or counted in the age interval [15, 20] and is not divided into other sub-dimensions such as [21, 25].

[0107] Correspondingly, for such a dimension containing only mutually exclusive sub-dimensions, step 203 or 303 calculates the data fluctuation measure value corresponding to the current data of each sub-dimension under each sub-dimension, which can specifically include:

[0108] The data difference value generated by the current data of the dimension corresponding to the dimension containing only mutually exclusive sub-dimensions with respect to the current data of the same dimension from the historical statistical data. For example, the difference between the number of users obtained by the age dimension on last Thursday and the number of users obtained by the age dimension on this Thursday, so that the root cause dimension can be determined according to the difference.

[0109] As can be seen from the above description, the method of the application can be well adapted to various complex business scenarios, for example, the case where the data between the dimension values is not mutually exclusive, i.e. the data has intersections. Here is another example:

[0110] Suppose the total DAU from all over the country on the day decreases from 1000 to 900, a decrease of 100. Analyze the sub-dimensions, i.e. the following sources, the DAU of source 1 (a certain province) decreases from 200 to 190, a decrease of 10, and after removing the influence of the user data of source 1 (i.e. the current DAU of source 1 is still counted as 200), the total DAU is 902.

[0111] According to the original Adtributor contribution degree calculation formula, its contribution degree is 10 / 100 = 10%, which is obviously unreasonable, because a user may exist in source 1 and source 2 at the same time, so the real contribution degree may be much less than 10%.

[0112] According to the contribution degree calculation formula of the application, its contribution degree is (900-902) / (900-1000) = 2%.

[0113] From the above example, it can be seen that the contribution degree calculated by the Adtributor algorithm is very different from the contribution degree of the application.

[0114] Similarly, for the abnormality degree calculation, assuming that the reference DAU and the current DAU of source 1 are both 100, the abnormality degree calculated by the Adtributor algorithm is 0, which represents no abnormality. However, in practice, considering the sharing of specific users and other source dimension values, there will be a large change, and there will be a large gap between the prior probability and the posterior probability, that is, the abnormality degree is large.

[0115] In summary, the data fluctuation attribution analysis method of the present application can be widely applied to various practical application scenarios, and is helpful for efficiently troubleshooting the root cause of data fluctuation, thereby promoting the elimination of subsequent work processes such as data error, and improving the accuracy and timeliness of data statistics.

[0116] Referring to Figure 6 The second aspect of the present application provides an embodiment of a data fluctuation attribution analysis system, which comprises an acquisition unit 601, a calculation unit 602 and a determination unit 603.

[0117] For the target statistical data obtained by multi-dimensional data summary analysis, the acquisition unit 601 is configured to acquire the sub-dimension current data participating in statistics under each sub-dimension of each dimension, wherein the sub-dimension current data of at least part of the dimensions belongs to different sub-dimensions under the same dimension.

[0118] The acquisition unit 601 is further configured to acquire historical statistical data different from the target statistical data acquisition time.

[0119] For each dimension in the multi-dimension, the calculation unit 602 is configured to calculate, according to the target statistical data and the historical statistical data, a data fluctuation metric value corresponding to the sub-dimension current data under each sub-dimension of the dimension, the data fluctuation metric value being used to represent the data change of the sub-dimension current data relative to the historical statistical data under the participation of other sub-dimension current data of the same dimension.

[0120] The determination unit 603 is configured to determine, according to the calculated data fluctuation metric values, a root cause dimension causing the target statistical data to fluctuate beyond a threshold value relative to the historical statistical data from the multi-dimension.

[0121] Optionally, if the data fluctuation metric value includes the probability distribution similarity corresponding to the sub-dimension current data and the index weight corresponding to the sub-dimension current data, the calculation unit 602 is specifically configured to:

[0122] For each sub-dimension under the dimension, the sub-dimension current data under the sub-dimension is counted, and the sub-dimension experience quantity of the sub-dimension current data appearing in different sub-dimensions of the same dimension is obtained.

[0123] The prior probability of the sub-dimension experience quantity relative to the historical statistical data and the posterior probability of the sub-dimension experience quantity relative to the target statistical data are used to represent the probability distribution of the statistical object in the statistical data at different collection times.

[0124] The probability distribution similarity between the prior probability and the posterior probability is calculated.

[0125] The sub-dimension current data corresponding to the sub-dimension is replaced with the corresponding sub-dimension historical data from the historical statistical data.

[0126] According to the sub-dimension historical data obtained by replacing, the index weight of the sub-dimension current data relative to the change quantity of the statistical data before and after is calculated.

[0127] Optionally, if the data fluctuation measure value includes the probability distribution similarity and the index weight, the determining unit 603 is specifically configured to:

[0128] For each dimension, it is judged whether there is a target number of sub-dimensions under the dimension, each type of data fluctuation measure value is greater than the corresponding data fluctuation threshold value, and the data fluctuation threshold value includes the probability similarity threshold value and the weight threshold value.

[0129] If there is, the dimension is determined as a root cause dimension.

[0130] Optionally, if there are multiple root cause dimensions determined, the system further includes:

[0131] The processing unit 604 is configured to sort the one type of data fluctuation measure value corresponding to each root cause dimension in descending order, and select a preset number of root cause dimensions ranked first as the final root cause dimensions.

[0132] Optionally, if there are multiple root cause dimensions determined, the system further includes a processing unit 604, which is configured to:

[0133] Each type of data fluctuation measure value corresponding to each root cause dimension is fused to obtain a result fusion value corresponding to each root cause dimension.

[0134] A preset number of root cause dimensions with a result fusion value exceeding a selection threshold value are selected as the final root cause dimensions.

[0135] Optionally, the system further includes a processing unit 604, which is configured to:

[0136] For each dimension in the multi-dimension, if the sub-dimensions under the dimension satisfy the mutual exclusion relationship, the dimension is regarded as a dimension containing only mutually exclusive sub-dimensions, and the data fluctuation measure value includes a data difference value corresponding to the current data of the sub-dimension; the mutual exclusion relationship indicates that the current data of each sub-dimension under the same dimension only belongs to one of the sub-dimensions;

[0137] The computing unit 602 is specifically configured to:

[0138] For the dimension containing only mutually exclusive sub-dimensions, the data difference value generated by the current data of the dimension relative to the current data of the same dimension from the historical statistical data is calculated.

[0139] Optionally, the system further includes a processing unit 604, configured to:

[0140] If the data fluctuation of the target statistical data relative to the historical statistical data exceeds the preset fluctuation range, the target statistical data is regarded as the statistical data triggering the timing attribution analysis, and whether the target statistical data is completed is queried through the data scheduling platform before the target statistical data starts the timing attribution analysis.

[0141] If not, continue to collect data from the data source through the data scheduling platform until the collection is completed.

[0142] In the embodiments of the present application, the operations performed by each unit of the data fluctuation attribution analysis system are similar to the operations described in the foregoing first aspect or any specific method embodiment of the first aspect, and will not be repeated here.

[0143] Referring to Figure 7 The data fluctuation attribution analysis device 700 in the embodiments of the present application can include one or more central processing units (CPU, central processing units) 701 and a memory 705, and the memory 705 stores one or more application programs or data.

[0144] The memory 705 can be volatile storage or persistent storage. The program stored in the memory 705 can include one or more modules, and each module can include a series of instruction operations in the data fluctuation attribution analysis device. Further, the central processing unit 701 can be configured to communicate with the memory 705 and execute a series of instruction operations in the memory 705 on the data fluctuation attribution analysis device 700.

[0145] The data fluctuation attribution analysis apparatus 700 can further include one or more power supplies 702, one or more wired or wireless network interfaces 703, one or more input / output interfaces 704, and / or one or more operating systems, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0146] The central processor 701 can perform operations performed by the first aspect or any of the specific method embodiments of the first aspect, and details are not repeated.

[0147] It can be understood that, in various embodiments of the present application, the sequence of steps does not mean the order of execution, the execution order of the steps should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0148] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0149] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. For example, the apparatus embodiments described above are only schematic. For example, the division of the units is only a logical function division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another system or apparatus, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, apparatus or unit, and can be electrical, mechanical or other forms.

[0150] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e. they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0151] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0152] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product (computer program product) is stored in a storage medium, includes a plurality of instructions to make a computer device (which can be a personal computer, a business server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, read-only memory), a random access memory (RAM, random access memory), a magnetic disk or an optical disk, and various media that can store program codes.

Claims

1. A method of data fluctuation attribution analysis, the method comprising: The method comprises the following steps: For target statistical data obtained by multi-dimensional data summary analysis, obtain sub-dimension current data of each sub-dimension of each dimension participating in statistics, wherein at least part of the sub-dimension current data of each dimension belongs to different sub-dimensions of the same dimension; Obtain historical statistical data different from the target statistical data collection time; For each dimension in the multi-dimensional data, calculate a data fluctuation metric value corresponding to the sub-dimension current data of each sub-dimension of the dimension according to the target statistical data and the historical statistical data, wherein the data fluctuation metric value is used to represent data changes of the sub-dimension current data relative to the historical statistical data under the participation of other sub-dimension current data of the same dimension; if the data fluctuation metric value comprises a probability distribution similarity corresponding to the sub-dimension current data and an index weight corresponding to the sub-dimension current data, the calculation of the data fluctuation metric value corresponding to the sub-dimension current data of each sub-dimension of the dimension comprises the following steps: for each sub-dimension of the dimension, count statistical objects to which the sub-dimension current data of the sub-dimension points, i.e., sub-dimension experience quantities obtained due to the fact that the same dimension belongs to different sub-dimensions; calculate a prior probability of the sub-dimension experience quantity relative to the historical statistical data and a posterior probability of the sub-dimension experience quantity relative to the target statistical data, wherein the prior probability and the posterior probability are used to represent probability distributions of the statistical objects in statistical data of different collection times; calculate the probability distribution similarity between the prior probability and the posterior probability; replace the sub-dimension current data corresponding to the sub-dimension with historical data corresponding to the sub-dimension from the historical statistical data; and calculate the index weight of the change quantity of the sub-dimension current data relative to the statistical data before and after replacement according to the historical data corresponding to the sub-dimension obtained by replacement; According to the calculated data fluctuation metric values, determine a root cause dimension from the multi-dimensional data, wherein the root cause dimension causes a fluctuation of the target statistical data relative to the historical statistical data to exceed a threshold value.

2. The data fluctuation attribution analysis method according to claim 1, characterized by, If the data fluctuation metric value is a multi-type data fluctuation metric value comprising a probability distribution similarity and an index weight, the determination of the root cause dimension from the multi-dimensional data comprises the following steps: For each dimension, determine whether there are target number of sub-dimensions of the dimension, each type of data fluctuation metric value of which is greater than a corresponding data fluctuation threshold value, wherein the data fluctuation threshold value comprises a probability similarity threshold value and a weight threshold value; If the answer is yes, determine that the dimension is the root cause dimension.

3. The data fluctuation attribution analysis method according to claim 1, characterized by, If there are multiple root cause dimensions determined, the method further comprises the following steps: Sort the one type of data fluctuation metric value corresponding to each root cause dimension in descending order, and select a preset number of root cause dimensions ranked first as the final root cause dimensions.

4. The data fluctuation attribution analysis method according to claim 1, characterized by, If there are multiple root cause dimensions determined, the method further comprises the following steps: Fuse each type of data fluctuation metric value corresponding to each root cause dimension to obtain a result fusion value corresponding to each root cause dimension; Select a preset number of root cause dimensions whose result fusion values exceed a selection threshold value as the final root cause dimensions.

5. The data fluctuation attribution analysis method according to claim 1, characterized by, Before obtaining the sub-dimension current data of each sub-dimension participating in statistics of each dimension, the method further comprises: For each dimension in the multiple dimensions, if the sub-dimensions under the dimension satisfy a mutual exclusion relationship, the dimension is regarded as a dimension containing only mutually exclusive sub-dimensions, and the data fluctuation measure value comprises a data difference value corresponding to the sub-dimension current data; the mutual exclusion relationship indicates that the sub-dimension current data under the same dimension only belongs to one sub-dimension; The data fluctuation measure value corresponding to the sub-dimension current data of each sub-dimension of the dimension is calculated, comprising: For the dimension containing only mutually exclusive sub-dimensions, the data difference value generated by the dimension current data corresponding to the dimension relative to the dimension current data of the same dimension from the historical statistical data is calculated.

6. The data fluctuation attribution analysis method according to claim 1, characterized by, Before obtaining the sub-dimension current data of each sub-dimension participating in statistics of each dimension, the method further comprises: If the data fluctuation of the target statistical data relative to the historical statistical data exceeds a preset fluctuation range, the target statistical data is regarded as statistical data triggering the timing attribution analysis, and whether the target statistical data is completed is queried through the data scheduling platform before the target statistical data starts the timing attribution analysis; If not, continue to collect data from the data source through the data scheduling platform until the collection is completed.

7. A data fluctuation attribution analysis system, characterized by, Comprise: An obtaining unit, a calculating unit and a determining unit; For the target statistical data obtained by the multi-dimension data summary analysis, the obtaining unit is configured to obtain the sub-dimension current data of each sub-dimension participating in statistics of each dimension, wherein the sub-dimension current data of at least part of the dimensions belongs to different sub-dimensions under the same dimension; The obtaining unit is further configured to obtain historical statistical data different from the target statistical data collection time; For each dimension in the multi-dimension, the computing unit is configured to calculate, according to the target statistical data and the historical statistical data, a data fluctuation metric value corresponding to the sub-dimension current data under each sub-dimension of the dimension, the data fluctuation metric value being used to represent data change of the sub-dimension current data relative to the historical statistical data with participation of other sub-dimension current data of the same dimension; if the data fluctuation metric value includes a probability distribution similarity corresponding to the sub-dimension current data and an index weight corresponding to the sub-dimension current data, the computing of the data fluctuation metric value corresponding to the sub-dimension current data under each sub-dimension of the dimension includes: for each sub-dimension of the dimension, counting a statistical object to which the sub-dimension current data under the sub-dimension points, a sub-dimension experience quantity obtained due to occurrence in different sub-dimensions of the same dimension; calculating a prior probability of the sub-dimension experience quantity relative to the historical statistical data and a posterior probability of the sub-dimension experience quantity relative to the target statistical data, the prior probability and the posterior probability being used to represent probability distributions of the statistical object in statistical data of different collection times; calculating the probability distribution similarity between the prior probability and the posterior probability; replacing the sub-dimension current data corresponding to the sub-dimension with corresponding sub-dimension historical data under the same sub-dimension from the historical statistical data; and calculating, according to the replaced sub-dimension historical data, the index weight of the sub-dimension current data relative to the change quantity of the two statistical data before and after replacement. The determining unit is configured to determine, according to the calculated data fluctuation metric values, a root cause dimension that causes the target statistical data to have a fluctuation exceeding a threshold value relative to the historical statistical data from the multi-dimension.

8. A data fluctuation attribution analysis apparatus characterized by comprising: The method comprises: a central processing unit, a memory and an input / output interface; the memory is a transitory storage memory or a persistent storage memory; the central processing unit is configured to communicate with the memory and execute instruction operations in the memory to perform the method in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, instructions, when the instructions are run on a computer, cause the computer to perform the method in any one of claims 1 to 6.

Images