Data analysis method, device, electronic device and storage medium

Abstract

The present invention provides a data analysis method, device, electronic device and storage medium, belonging to the field of data analysis technology. The method includes: determining a target data set and a comparison data set from a corresponding business query list based on analysis requirements; determining the first overall deviation corresponding to each dimension in the target data set based on the target data set and the comparison data set, the first overall deviation characterizing the difference in each dimension of the target data set relative to the comparison data set; when the largest first overall deviation is less than or equal to a preset threshold, outputting the analysis result; when the largest first overall deviation is greater than the preset threshold, performing a drill-down analysis on the dimension corresponding to the largest first overall deviation, and outputting the analysis result based on the drill-down analysis content. The data analysis method can automatically analyze the data, save human resources, and has a fast analysis speed, saving time.

Description

Technical Field

[0001] The present invention relates to the field of data analysis technology, and in particular to a data analysis method, device, electronic equipment and storage medium. Background Art

[0002] With the development of the Internet, the value of data has become particularly prominent. All walks of life cannot do without the support of data. However, faced with complex and massive data, it is not easy to extract key information from it. Professional data analysts are often required to display and summarize key information in the data through data cleaning, visualization, feature extraction, attribution, etc. However, because there is no universal analysis template or standardized analysis method, the analysis process cannot be well migrated in different data scenarios, which undoubtedly increases the workload of analysts. Especially when encountering sudden situations, such as a large change in a certain business indicator, various failures causing abnormal fluctuations in indicators, etc., it is often necessary to temporarily arrange analysts for analysis, which consumes a lot of manpower and cannot analyze the cause in time.

[0003] At present, there are some software that can assist in analysis on the market, but these software are still designed for analysts and some auxiliary functions are developed to facilitate their operations. They still require human resources and cannot meet the needs of business personnel without professional background knowledge. Summary of the invention

[0004] The present invention provides a data analysis method, device, electronic device and storage medium, which are used to solve the defect that data analysis in the prior art requires professional analysts and consumes human resources, and realize automatic data analysis according to analysis requirements, saving human resources.

[0005] The present invention provides a data analysis method, comprising:

[0006] Determine the target data set and the comparison data set from the corresponding business query list based on the analysis requirements;

[0007] Determine, based on the target data set and the comparison data set, first overall deviations corresponding to respective dimensions in the target data set, wherein the first overall deviations represent differences between the target data set and the comparison data set in respective dimensions;

[0008] When the largest first overall deviation is less than or equal to a preset threshold, outputting the analysis result;

[0009] When the largest first overall deviation is greater than a preset threshold, a drill-down analysis is performed on the dimension corresponding to the largest first overall deviation, and an analysis result is output based on the drill-down analysis content.

[0010] According to a data analysis method provided by the present invention, the drill-down analysis on the dimension corresponding to the largest first overall deviation includes:

[0011] Determine a first drill-down dimension for drill-down analysis, and perform drill-down analysis on a dimension corresponding to the largest first overall deviation based on the first drill-down dimension;

[0012] During the drill-down analysis, a second overall deviation corresponding to each of the first drill-down dimensions is determined based on the target data set and the comparison data set;

[0013] When the second largest overall deviation is less than or equal to the preset threshold, stop the drill-down analysis;

[0014] When the largest second overall deviation is greater than the preset threshold, a second drill-down dimension for drill-down analysis is determined, and a drill-down analysis is performed on the first drill-down dimension corresponding to the largest second overall deviation based on the second drill-down dimension until the N+1th overall deviations corresponding to each Nth drill-down dimension are less than or equal to the preset threshold, and then the drill-down analysis is stopped, where N is greater than or equal to 2.

[0015] According to a data analysis method provided by the present invention, before determining the target data set and the comparison data set from the corresponding business query list based on the analysis requirements, the method further includes:

[0016] Automatically collect the detailed data of the buried points corresponding to each business in real time, clean the collected detailed data of the buried points and store them in the underlying data tables corresponding to each business;

[0017] The data in the underlying data table corresponding to each of the businesses are extracted respectively to obtain important indicator data of each of the businesses, and business query lists corresponding to each of the businesses are generated based on the important indicator data of each of the businesses.

[0018] According to a data analysis method provided by the present invention, determining the first overall deviation corresponding to each dimension in the target data set based on the target data set and the comparison data set includes:

[0019] Based on the distribution corresponding to each classification under each dimension in the target data set and the classification index corresponding to each classification under each dimension in the comparison data set, calculate the first joint index corresponding to each dimension in the target data set;

[0020] Based on the distribution and classification index corresponding to each classification under each dimension in the comparison data set, calculate the second joint index corresponding to each dimension in the comparison data set;

[0021] Based on each of the first joint indicators and each of the second joint indicators, a first overall deviation corresponding to each dimension in the target data set is determined.

[0022] According to a data analysis method provided by the present invention, the first joint index corresponding to each dimension in the target data set is calculated based on the distribution corresponding to each classification under each dimension in the target data set and the classification index corresponding to each classification under each dimension in the comparison data set, including:

[0023] For each dimension in the target data set, the distribution corresponding to each category under the dimension and the classification index corresponding to each category in the comparison data set are multiplied respectively to obtain the first joint index corresponding to each dimension in the target data set.

[0024] According to a data analysis method provided by the present invention, the second joint index corresponding to each dimension in the comparison data set is calculated based on the distribution and classification index corresponding to each classification under each dimension in the comparison data set, including:

[0025] For each dimension in the comparison data set, the distribution and classification index corresponding to each classification under the dimension are multiplied respectively to obtain the second joint index corresponding to each dimension in the comparison data set.

[0026] According to a data analysis method provided by the present invention, determining first overall deviations corresponding to respective dimensions in the target data set based on the first joint indicators and the second joint indicators includes:

[0027] Subtracting each of the first joint indicators from the corresponding second joint indicators to obtain an initial overall deviation of the target data set in each dimension;

[0028] The absolute value of the initial overall deviation corresponding to each dimension of the target data set is respectively used as the first overall deviation corresponding to each dimension in the target data set.

[0029] The present invention also provides a data analysis device, comprising:

[0030] A data module is used to determine a target data set and a comparison data set from a corresponding business query list based on analysis requirements;

[0031] an analysis module, configured to determine, based on the target data set and the comparison data set, first overall deviations corresponding to respective dimensions in the target data set, wherein the first overall deviations represent differences between the target data set and the comparison data set in respective dimensions; and when the largest first overall deviation is greater than a preset threshold, perform a drill-down analysis on the dimension corresponding to the largest first overall deviation;

[0032] The output module is used to output the analysis result based on the drill-down analysis content, or output the analysis result when the largest first overall deviation is less than or equal to a preset threshold.

[0033] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the data analysis methods described above is implemented.

[0034] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the data analysis method described in any one of the above is implemented.

[0035] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the data analysis methods described above.

[0036] The data analysis method, device, electronic device and storage medium provided by the present invention automatically analyze data according to analysis needs, and output analysis results based on the analysis conditions in various dimensions of the target data set. The method can automatically analyze data by only determining the analysis needs, and does not require high professional background knowledge of personnel. No analyst is required to perform analysis, which saves human resources. In addition, the method has a fast analysis speed, which greatly shortens the time of data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 is a flow chart of a data analysis method provided by an embodiment of the present invention;

[0039] Figure 2 It is one of the structural schematic diagrams of the data analysis device provided by the embodiment of the present invention;

[0040] Figure 3 This is the second structural diagram of the data analysis device provided by the embodiment of the present invention;

[0041] Figure 4 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] In the existing technology, it is not easy to extract key information from the data. It often requires professional data analysts to process the data in various ways to display and summarize the key information in the data. However, because there is no universal analysis template or standardized analysis method, the analysis process cannot be well migrated in different data scenarios, which undoubtedly increases the workload of analysts and consumes a lot of manpower. Although some software that can assist in analysis has been born on the market, these software are still designed for analysts, still require human resources, and cannot meet the needs of business personnel without professional background knowledge.

[0044] In view of the above problems, an embodiment of the present invention provides a data analysis method. Figure 1 is a flow chart of a data analysis method provided by an embodiment of the present invention, such as Figure 1 As shown in , the method includes:

[0045] S101: Determine a target data set and a comparison data set from a corresponding business query list based on analysis requirements;

[0046] S102: Determine, based on the target data set and the comparison data set, first overall deviations corresponding to respective dimensions in the target data set, wherein the first overall deviations represent differences between the target data set and the comparison data set in respective dimensions;

[0047] S103: When the maximum first overall deviation is less than or equal to a preset threshold, outputting the analysis result;

[0048] S104: When the largest first overall deviation is greater than a preset threshold, a drill-down analysis is performed on a dimension corresponding to the largest first overall deviation, and an analysis result is output based on the drill-down analysis content.

[0049] Specifically, Figure 2As shown in , the analysis requirement can be generated after the monitoring alarm device monitors the report abnormality, or it can be output by the user. Furthermore, after the monitoring alarm device generates the first analysis requirement, the user can further input the second analysis requirement required by the user, such as setting the analysis dimension, setting the analysis time, and setting the special dimension that the user wants to know, etc. The subsequent data analysis is based on the first analysis requirement generated by the monitoring alarm device, and refers to the second analysis requirement of the user for data analysis. The first analysis requirement and the second analysis requirement constitute the total analysis requirement.

[0050] Based on the analysis requirements, the target data set and the comparison data set are determined from the corresponding business query list. The target data set is the data set that needs to be analyzed, and the comparison data set is the data set selected for comparison with the target data set. For example, if the analysis requirement is "analyze the user login status of business A in this month and the same month last year", the business query list is the business query list corresponding to business A, the target data set is the data set that represents the login status of business A in this month, and the comparison data set is the data set that represents the user login status of business A in the same month last year.

[0051] After obtaining the target data set and the comparison data set, the first total deviation corresponding to each dimension in the target data set is determined according to the target data set and the comparison data set. The first total deviation can represent the difference between the target data set and the comparison data set in each dimension. The first total deviation corresponding to each dimension is determined. When the largest first total deviation is less than or equal to the preset threshold, it indicates that the user login situation of business A in this month this year and the same month last year has not changed much from the data point of view. At this time, the analysis result is output. The analysis result may include the first total deviation value of each dimension, the analysis conclusion generated based on the first total deviation value of each dimension, etc. When the largest first overall deviation is greater than the preset threshold, it indicates that the user login situation of business A in this month this year and the same month last year has changed greatly in some dimensions from the data point of view. At this time, the dimension corresponding to the largest first overall deviation is drilled down for analysis, and the analysis result is output based on the drill-down analysis content. The analysis result may include the overall situation based on the drill-down analysis content in each dimension and the generated analysis conclusion, etc., and of course, it may also include other content, which is not specifically limited here. The way to output the analysis result may be to display it on a display screen, and the analysis result may be intuitively displayed in the form of a table plus text.

[0052] The data analysis method provided by the present invention automatically analyzes data according to analysis needs, and outputs analysis results based on the analysis conditions in various dimensions of the target data set. The method can automatically analyze data by only determining the analysis needs. The operation is simple and does not require high professional background knowledge of personnel. No analyst is required to perform the analysis, thus saving human resources. In addition, the method has a fast analysis speed, which greatly shortens the time for data analysis.

[0053] In one embodiment, the drill-down analysis on the dimension corresponding to the largest first overall deviation includes:

[0054] Determine a first drill-down dimension for drill-down analysis, and perform drill-down analysis on a dimension corresponding to the largest first overall deviation based on the first drill-down dimension;

[0055] During the drill-down analysis, a second overall deviation corresponding to each of the first drill-down dimensions is determined based on the target data set and the comparison data set;

[0056] When the second largest overall deviation is less than or equal to the preset threshold, stop the drill-down analysis;

[0057] When the largest second overall deviation is greater than the preset threshold, a second drill-down dimension for drill-down analysis is determined, and a drill-down analysis is performed on the first drill-down dimension corresponding to the largest second overall deviation based on the second drill-down dimension until the N+1th overall deviations corresponding to each Nth drill-down dimension are less than or equal to the preset threshold, and then the drill-down analysis is stopped, where N is greater than or equal to 2.

[0058] Specifically, the method for performing drill-down analysis on the dimension corresponding to the largest first overall deviation may include:

[0059] Determine the first drill-down dimension for drill-down analysis. For example, the dimensions corresponding to the target data set include age, gender, city, and occupation. The dimension corresponding to the largest first overall deviation is the age dimension. At this time, the remaining dimensions except the age dimension are sorted in descending order according to the first overall deviation. After sorting, the dimension with a preset percentage ranking at the top is used as the first drill-down dimension, where the preset percentage can be pre-set according to the analysis needs and is not specifically limited here.

[0060] After the first drill-down dimension is determined, a drill-down analysis is performed on the dimension corresponding to the largest first overall deviation based on the first drill-down dimension. Exemplarily, for example, the first drill-down dimension determined based on a preset percentage includes gender and city, and the age category corresponding to the largest first overall deviation includes youth, middle-aged, and elderly. At this time, the gender dimension and the city dimension are analyzed separately under the youth category, the gender dimension and the city dimension are analyzed separately under the middle-aged category, and the gender dimension and the city dimension are analyzed separately under the elderly category.

[0061] When performing drill-down analysis, the second overall deviation corresponding to each first drill-down dimension is determined based on the target data set and the comparison data set. Exemplarily, a youth target data set corresponding to youth, a middle-aged target data set corresponding to middle-aged, and an elderly target data set corresponding to the elderly under the age dimension are obtained from the target data set, and a youth comparison data set corresponding to youth, a middle-aged comparison data set corresponding to middle-aged, and an elderly comparison data set corresponding to the elderly under the age dimension are obtained from the comparison data set. Based on the youth target data set, the middle-aged target data set, the elderly target data set, the youth comparison data set, the middle-aged comparison data set, and the elderly comparison data set, the second overall deviation corresponding to each first drill-down dimension, namely, gender and city, is determined.

[0062] When the largest second overall deviation is less than or equal to the preset threshold, the drill-down analysis is stopped. When the largest second overall deviation is greater than the preset threshold, the second drill-down dimension for drill-down analysis is determined, and the first drill-down dimension corresponding to the largest second overall deviation is drilled down based on the second drill-down dimension, until the N+1th overall deviations corresponding to each Nth drill-down dimension are all less than or equal to the preset threshold, the drill-down analysis is stopped, where N is greater than or equal to 2. That is, when the largest third overall deviation corresponding to the second drill-down dimension is less than or equal to the preset threshold, the drill-down analysis is stopped. When the largest third overall deviation corresponding to the second drill-down dimension is greater than the preset threshold, the third drill-down dimension for drill-down analysis is determined, and the second drill-down dimension corresponding to the largest third overall deviation is drilled down based on the third drill-down dimension, and this is repeated until the overall deviations corresponding to a certain round of drill-down dimensions are all less than or equal to the preset threshold, and the drill-down analysis is stopped.

[0063] In one embodiment, before determining the target data set and the comparison data set from the corresponding business query list based on the analysis requirements, the method further includes:

[0064] Automatically collect the detailed data of the buried points corresponding to each business in real time, clean the collected detailed data of the buried points and store them in the underlying data tables corresponding to each business;

[0065] The data in the underlying data table corresponding to each of the businesses are extracted respectively to obtain important indicator data of each of the businesses, and business query lists corresponding to each of the businesses are generated based on the important indicator data of each of the businesses.

[0066] Specifically, it can be understood that the basis for data analysis is to obtain a large amount of data. Therefore, before determining the target data set and the comparison data set from the corresponding business query list based on the analysis requirements, it also includes:

[0067] Automatically collect the detailed data of the buried points corresponding to each business in real time and clean the collected detailed data of the buried points. Common data cleaning methods include: duplicate value processing, vacancy value processing, abnormal value processing and data standardization. The cleaned detailed data of the buried points are stored in the underlying data table corresponding to each business based on the corresponding business. Since the amount of data corresponding to the business may be very large, excessive data volume will increase the time and difficulty of calculation. Therefore, data extraction can be performed on the data in the underlying data table corresponding to each business, for example, only extracting the important indicator data of each business. After data extraction, the important indicator data of the same business are stored in the business query list corresponding to the business.

[0068] In one embodiment, determining the first overall deviation corresponding to each dimension in the target data set based on the target data set and the comparison data set includes:

[0069] Based on the distribution corresponding to each classification under each dimension in the target data set and the classification index corresponding to each classification under each dimension in the comparison data set, calculate the first joint index corresponding to each dimension in the target data set;

[0070] Based on the distribution and classification index corresponding to each classification under each dimension in the comparison data set, calculate the second joint index corresponding to each dimension in the comparison data set;

[0071] Based on each of the first joint indicators and each of the second joint indicators, a first overall deviation corresponding to each dimension in the target data set is determined.

[0072] Specifically, after obtaining the target data set and the comparison data set, determining the first overall deviation corresponding to each dimension in the target data set based on the target data set and the comparison data set may include:

[0073] First, the first joint index corresponding to each dimension in the target data set is calculated according to the distribution corresponding to each classification under each dimension in the target data set and the classification index corresponding to each classification under each dimension in the comparison data set. For example, the dimensions in the target data set may include age, gender, city, and occupation, and the classification of the age dimension may include youth, middle-aged, and elderly, and the classification of the gender dimension may include male and female.

[0074] Secondly, the second joint index corresponding to each dimension in the comparison data set is calculated according to the distribution and classification index corresponding to each classification under each dimension in the comparison data set. For example, the dimensions in the comparison data set may include age, gender, city and occupation, the classification of the age dimension may include youth, middle-aged and old age, and the classification of the gender dimension may include male and female.

[0075] After determining each first joint indicator and each second joint indicator, the first overall deviation corresponding to each dimension in the target data set can be determined according to each first joint indicator and each second joint indicator.

[0076] In the above embodiment, during data analysis, data distribution changes and classification index changes are combined for data analysis, taking into account the impact of distribution changes and classification index changes, and the cause of the problem can be located more accurately through analysis.

[0077] In one embodiment, the first joint index corresponding to each dimension in the target data set is calculated based on the distribution corresponding to each category under each dimension in the target data set and the classification index corresponding to each category under each dimension in the comparison data set, including:

[0078] For each dimension in the target data set, the distribution corresponding to each category under the dimension and the classification index corresponding to each category in the comparison data set are multiplied respectively to obtain the first joint index corresponding to each dimension in the target data set.

[0079] Specifically, for each dimension in the target data set, the first joint index corresponding to each dimension is obtained by multiplying the distribution corresponding to each category under each dimension in the target data set with the classification index corresponding to each category in the comparison data set, and then adding the calculation results corresponding to each category.

[0080] Exemplarily, the calculation method of the first joint indicator corresponding to the age dimension in the target data set is taken as an example to explain in detail:

[0081] For example, for the age dimension in the target data set, the subcategories include youth, middle-aged, and elderly. The distribution of youth accounts for 49%, and the corresponding classification indicators for youth, such as login indicators, are 30%. The distribution of middle-aged accounts for 37%, and the corresponding classification indicators for middle-aged, such as login indicators, are 35%. The distribution of elderly accounts for 14%, and the corresponding classification indicators for elderly, such as login indicators, are 27%. For example, for the age dimension in the comparison data set, the subcategories include youth, middle-aged, and elderly. The distribution of youth accounts for 38%, and the corresponding classification indicators for youth, such as login indicators, are 30%. The distribution of middle-aged accounts for 47%, and the corresponding classification indicators for middle-aged, such as login indicators, are 35%. The distribution of elderly accounts for 15%, and the corresponding classification indicators for elderly, such as login indicators, are 28%.

[0082] For the age dimension in the target data set, the corresponding first joint index T1 is: T1=49%×30%+37%×35%+14%×28%.

[0083] In one embodiment, the calculating the second joint index corresponding to each dimension in the comparison data set based on the distribution and classification index corresponding to each classification under each dimension in the comparison data set includes:

[0084] For each dimension in the comparison data set, the distribution and classification index corresponding to each classification under each dimension are multiplied respectively to obtain the second joint index corresponding to each dimension in the comparison data set.

[0085] Exemplarily, the following is a detailed description of the calculation method of the second joint indicator corresponding to the age dimension in the comparison data set as an example:

[0086] For example, for the age dimension in the comparison data set, the categories include youth, middle-aged and elderly. The distribution of youth accounts for 38%, and the corresponding classification indicators of youth, such as login indicators, are 30%. The distribution of middle-aged accounts for 47%, and the corresponding classification indicators of middle-aged, such as login indicators, are 35%. The distribution of elderly accounts for 15%, and the corresponding classification indicators of elderly, such as login indicators, are 28%.

[0087] For the age dimension in the comparison data set, the second joint indicator T2 corresponding to the age dimension is: T2=38%×30%+47%×35%+15%×28%.

[0088] In one embodiment, determining first overall deviations corresponding to respective dimensions in the target data set based on the first joint indicators and the second joint indicators includes:

[0089] Subtracting each of the first joint indicators from the corresponding second joint indicators to obtain an initial overall deviation of the target data set in each dimension;

[0090] The absolute value of the initial overall deviation corresponding to each dimension of the target data set is respectively used as the first overall deviation corresponding to each dimension in the target data set.

[0091] Specifically, after obtaining each first joint indicator and each second joint indicator, a method for determining the first overall deviation corresponding to each dimension in the target data set according to each first joint indicator and each second joint indicator includes:

[0092] The difference between each first joint indicator and the corresponding second joint indicator is respectively made to obtain the initial overall deviation of the target data set in each dimension. The initial overall deviation may be positive or negative. Therefore, the absolute value of the initial overall deviation corresponding to each dimension of the target data set is taken as the first overall deviation corresponding to each dimension in the target data set, so as to facilitate the subsequent comparison of the first overall deviation with the preset threshold.

[0093] Exemplarily, the following is a detailed description of the calculation method of the first overall deviation corresponding to the age dimension in the target data set as an example:

[0094] Based on the above embodiment, it can be known that the first joint indicator T1 corresponding to the age dimension is: T1=49%×30%+37%×35%+14%×28%; the corresponding second joint indicator T2 is: T2=38%×30%+47%×35%+15%×28%.

[0095] Therefore, the initial overall deviation d1 corresponding to the age dimension in the target data set is: d1 = T1 - T2 = -0.48%.

[0096] Based on the initial overall deviation d1 corresponding to the age dimension, the first overall deviation D1 corresponding to the age dimension in the target data set is: D1=|d1|=0.48%.

[0097] The data analysis device 200 provided by the present invention is described below. The data analysis device 200 described below and the data analysis method described above can be referenced to each other.

[0098] like Figure 2 As shown in , the data analysis device 200 includes: a data module 210, an analysis module 220 and an output module 230;

[0099] The data module 210 is used to determine the target data set and the comparison data set from the corresponding business query list based on the analysis requirements;

[0100] The analysis module 220 is used to determine the first overall deviations corresponding to the dimensions in the target data set based on the target data set and the comparison data set, wherein the first overall deviations represent the differences between the target data set and the comparison data set in each dimension; when the largest first overall deviation is greater than a preset threshold, a drill-down analysis is performed on the dimension corresponding to the largest first overall deviation;

[0101] The output module 230 is used to output the analysis result based on the drill-down analysis content, or output the analysis result when the largest first overall deviation is less than or equal to a preset threshold.

[0102] Specifically, Figure 3 As shown in , this device can be a general data analysis device 200 for various services, and the data analysis device 200 includes: a data module 210, an analysis module 220 and an output module 230. The data analysis device 200 can be connected to a monitoring alarm device to obtain analysis requirements from the monitoring alarm device.

[0103] The data module 210 is the guarantee of the data resources of the entire device. The data module 210 is a module for automatic maintenance of data, including automatic and real-time collection of detailed data of various services, cleaning of detailed data of various services and automatic storage in the underlying data tables of various services, and further extraction of data in the underlying data tables, extracting important indicators for presentation and analysis of business concerns, etc. The data module 210 provides functions such as automatic data collection, cleaning and extraction, and provides data foundation guarantee for subsequent data analysis.

[0104] The analysis module 220 is the core of the entire data analysis device 200, including real-time acquisition of data extracted by the data module 210, analysis of data according to the design process, etc. The analysis module 220 is the most complex module. Data analysis is completed at this layer, which is the embodiment of the automation of the entire platform. After completing the data analysis, the analysis module 220 transmits the analysis results to the output module 230, which outputs them to the user.

[0105] The output module 230 mainly includes the optimization and layout of some output pages, including the selection method of the content of the analysis results, the display of the conclusion part, and the personalized display of the table, etc. The output module 230 user interacts with the data analysis device 200, that is, the user transmits the analysis requirements to the data analysis device 200, and the data analysis device 200 presents the analysis results to the user after layout.

[0106] In one embodiment, the analysis module 220 is further configured to:

[0107] Determine a first drill-down dimension for drill-down analysis, and perform drill-down analysis on a dimension corresponding to the largest first overall deviation based on the first drill-down dimension;

[0108] During the drill-down analysis, a second overall deviation corresponding to each of the first drill-down dimensions is determined based on the target data set and the comparison data set;

[0109] When the second largest overall deviation is less than or equal to the preset threshold, stop the drill-down analysis;

[0110] When the largest second overall deviation is greater than the preset threshold, a second drill-down dimension for drill-down analysis is determined, and a drill-down analysis is performed on the first drill-down dimension corresponding to the largest second overall deviation based on the second drill-down dimension, until the Nth overall deviation corresponding to each Nth drill-down dimension is less than or equal to the preset threshold, and the drill-down analysis is stopped, where N is greater than or equal to 2.

[0111] In one embodiment, the data module 210 is further used to:

[0112] Automatically collect the detailed data of the buried points corresponding to each business in real time, clean the collected detailed data of the buried points and store them in the underlying data tables corresponding to each business;

[0113] The data in the underlying data table corresponding to each of the businesses are extracted respectively to obtain important indicator data of each of the businesses, and business query lists corresponding to each of the businesses are generated based on the important indicator data of each of the businesses.

[0114] In one embodiment, the analysis module 220 is further configured to:

[0115] Based on the distribution corresponding to each classification under each dimension in the target data set and the classification index corresponding to each classification under each dimension in the comparison data set, calculate the first joint index corresponding to each dimension in the target data set;

[0116] Based on the distribution and classification index corresponding to each classification under each dimension in the comparison data set, calculate the second joint index corresponding to each dimension in the comparison data set;

[0117] Based on each of the first joint indicators and each of the second joint indicators, a first overall deviation corresponding to each dimension in the target data set is determined.

[0118] In one embodiment, the analysis module 220 is further configured to:

[0119] For each dimension in the target data set, the distribution corresponding to each category under the dimension and the classification index corresponding to each category in the comparison data set are multiplied respectively to obtain the first joint index corresponding to each dimension in the target data set.

[0120] In one embodiment, the analysis module 220 is further configured to:

[0121] For each dimension in the comparison data set, the distribution and classification index corresponding to each classification under the dimension are multiplied respectively to obtain the second joint index corresponding to each dimension in the comparison data set.

[0122] In one embodiment, the analysis module 220 is further configured to:

[0123] Subtracting each of the first joint indicators from the corresponding second joint indicators to obtain an initial overall deviation of the target data set in each dimension;

[0124] The absolute value of the initial overall deviation corresponding to each dimension of the target data set is respectively used as the first overall deviation corresponding to each dimension in the target data set.

[0125] The data analysis device provided by the present invention automatically analyzes data according to analysis requirements, and outputs analysis results based on the analysis conditions in various dimensions of the target data set. The method can automatically analyze data by only determining the analysis requirements, and does not require high professional background knowledge of personnel. In addition, the method does not require analysts to perform analysis and has a fast analysis speed, saving human resources and time costs.

[0126] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the data analysis method, which includes:

[0127] Determine the target data set and the comparison data set from the corresponding business query list based on the analysis requirements;

[0128] Determine, based on the target data set and the comparison data set, first overall deviations corresponding to respective dimensions in the target data set, wherein the first overall deviations represent differences between the target data set and the comparison data set in respective dimensions;

[0129] When the largest first overall deviation is less than or equal to a preset threshold, outputting the analysis result;

[0130] When the largest first overall deviation is greater than a preset threshold, a drill-down analysis is performed on the dimension corresponding to the largest first overall deviation, and an analysis result is output based on the drill-down analysis content.

[0131] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0132] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can perform the data analysis method provided by the above methods, the method comprising:

[0133] Determine the target data set and the comparison data set from the corresponding business query list based on the analysis requirements;

[0134] Determine, based on the target data set and the comparison data set, first overall deviations corresponding to respective dimensions in the target data set, wherein the first overall deviations represent differences between the target data set and the comparison data set in respective dimensions;

[0135] When the largest first overall deviation is less than or equal to a preset threshold, outputting the analysis result;

[0136] When the largest first overall deviation is greater than a preset threshold, a drill-down analysis is performed on the dimension corresponding to the largest first overall deviation, and an analysis result is output based on the drill-down analysis content.

[0137] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the data analysis method provided by the above methods, the method comprising:

[0138] Determine the target data set and the comparison data set from the corresponding business query list based on the analysis requirements;

[0139] Determine, based on the target data set and the comparison data set, first overall deviations corresponding to respective dimensions in the target data set, wherein the first overall deviations represent differences between the target data set and the comparison data set in respective dimensions;

[0140] When the largest first overall deviation is less than or equal to a preset threshold, outputting the analysis result;

[0141] When the largest first overall deviation is greater than a preset threshold, a drill-down analysis is performed on the dimension corresponding to the largest first overall deviation, and an analysis result is output based on the drill-down analysis content.

[0142] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0143] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data analysis method, characterized in that: include: Determine the target data set and the comparison data set from the corresponding business query list based on the analysis requirements; Determine, based on the target data set and the comparison data set, first overall deviations corresponding to respective dimensions in the target data set, wherein the first overall deviations represent differences between the target data set and the comparison data set in respective dimensions; When the largest first overall deviation is less than or equal to a preset threshold, outputting the analysis result; When the largest first overall deviation is greater than a preset threshold, a drill-down analysis is performed on the dimension corresponding to the largest first overall deviation, and an analysis result is output based on the drill-down analysis content; The drill-down analysis on the dimension corresponding to the largest first overall deviation includes: Determine a first drill-down dimension for drill-down analysis, and perform drill-down analysis on a dimension corresponding to the largest first overall deviation based on the first drill-down dimension; During the drill-down analysis, a second overall deviation corresponding to each of the first drill-down dimensions is determined based on the target data set and the comparison data set; When the second largest overall deviation is less than or equal to the preset threshold, stop the drill-down analysis; When the largest second overall deviation is greater than the preset threshold, a second drill-down dimension for drill-down analysis is determined, and drill-down analysis is performed on the first drill-down dimension corresponding to the largest second overall deviation based on the second drill-down dimension until the N+1th overall deviation corresponding to each Nth drill-down dimension is less than or equal to the preset threshold, and the drill-down analysis is stopped, where N is greater than or equal to 2; The determining of the first drill-down dimension for the drill-down analysis includes: For each dimension in the target data set, the dimensions except the dimension corresponding to the largest first overall deviation are sorted from large to small, and the dimensions with a preset percentage ranking at the top are determined as the first drill-down dimensions.

2. The data analysis method according to claim 1, characterized in that: Before determining the target data set and the comparison data set from the corresponding business query list based on the analysis requirements, the method further includes: Automatically collect the detailed data of the buried points corresponding to each business in real time, clean the collected detailed data of the buried points and store them in the underlying data tables corresponding to each business; The data in the underlying data table corresponding to each of the businesses are extracted respectively to obtain important indicator data of each of the businesses, and business query lists corresponding to each of the businesses are generated based on the important indicator data of each of the businesses.

3. The data analysis method according to claim 1, characterized in that: The determining, based on the target data set and the comparison data set, first overall deviations respectively corresponding to the dimensions in the target data set comprises: Based on the distribution corresponding to each classification under each dimension in the target data set and the classification index corresponding to each classification under each dimension in the comparison data set, calculate the first joint index corresponding to each dimension in the target data set; Based on the distribution and classification index corresponding to each classification under each dimension in the comparison data set, calculate the second joint index corresponding to each dimension in the comparison data set; Based on each of the first joint indicators and each of the second joint indicators, a first overall deviation corresponding to each dimension in the target data set is determined.

4. The data analysis method according to claim 3, characterized in that: The calculating, based on the distribution corresponding to each classification under each dimension in the target data set and the classification index corresponding to each classification under each dimension in the comparison data set, the first joint index corresponding to each dimension in the target data set includes: For each dimension in the target data set, the distribution corresponding to each category under the dimension and the classification index corresponding to each category in the comparison data set are multiplied respectively to obtain the first joint index corresponding to each dimension in the target data set.

5. The data analysis method according to claim 3, characterized in that: The calculating, based on the distribution and classification index corresponding to each classification under each dimension in the comparison data set, the second joint index corresponding to each dimension in the comparison data set includes: For each dimension in the comparison data set, the distribution and classification index corresponding to each classification under the dimension are multiplied respectively to obtain the second joint index corresponding to each dimension in the comparison data set.

6. The data analysis method according to any one of claims 3 to 5, characterized in that: Determining first overall deviations corresponding to respective dimensions in the target data set based on the first joint indicators and the second joint indicators includes: Subtracting each of the first joint indicators from the corresponding second joint indicators to obtain an initial overall deviation of the target data set in each dimension; The absolute value of the initial overall deviation corresponding to each dimension of the target data set is respectively used as the first overall deviation corresponding to each dimension in the target data set.

7. A data analysis device, characterized in that: include: A data module is used to determine a target data set and a comparison data set from a corresponding business query list based on analysis requirements; an analysis module, configured to determine, based on the target data set and the comparison data set, first overall deviations corresponding to respective dimensions in the target data set, wherein the first overall deviations represent differences between the target data set and the comparison data set in respective dimensions; and when the largest first overall deviation is greater than a preset threshold, perform a drill-down analysis on the dimension corresponding to the largest first overall deviation; An output module, used to output the analysis result based on the drill-down analysis content, or output the analysis result when the largest first overall deviation is less than or equal to a preset threshold; The analysis module is also specifically used for: Determine a first drill-down dimension for drill-down analysis, and perform drill-down analysis on a dimension corresponding to the largest first overall deviation based on the first drill-down dimension; During the drill-down analysis, a second overall deviation corresponding to each of the first drill-down dimensions is determined based on the target data set and the comparison data set; When the second largest overall deviation is less than or equal to the preset threshold, stop the drill-down analysis; When the largest second overall deviation is greater than the preset threshold, determine a second drill-down dimension for drill-down analysis, and perform drill-down analysis on the first drill-down dimension corresponding to the largest second overall deviation based on the second drill-down dimension until the Nth overall deviation corresponding to each Nth drill-down dimension is less than or equal to the preset threshold, and then stop the drill-down analysis, where N is greater than or equal to 2; The determining of the first drill-down dimension for the drill-down analysis includes: For each dimension in the target data set, the dimensions except the dimension corresponding to the largest first overall deviation are sorted from large to small, and the dimensions with a preset percentage ranking at the top are determined as the first drill-down dimensions.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the data analysis method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data analysis method according to any one of claims 1 to 6 is implemented.

Images