Intelligent dynamic dimension sorting method and system based on multi-dimensional feature parameters

Abstract

The present invention relates to the technical field of multidimensional data analysis in big data, and discloses an intelligent dynamic dimensional sorting method and system based on multidimensional feature parameters, the method comprising: creating a commodity hierarchical dimension table containing the atomic dimensions and hierarchical relationships of commodities according to the standard dimensional data of commodities stored in a data warehouse; creating a commodity parameter dimension table containing all parameter types and corresponding parameter values ​​of commodities according to the parameter types corresponding to each commodity and in combination with the hierarchical dimension table; disassembling commodities of different levels according to parameter types, and performing dimensionally differentiated sorting on each parameter value after disassembly through a sorting algorithm. The intelligent dynamic dimensional sorting method and system based on multidimensional feature parameters provided by the present invention solve the problem that traditional business multidimensional definitions cannot be controlled, reduce physical storage space and manual management costs through intelligent analysis and sorting, and can help business data analysts gain insight into key issues more quickly.

Description

Technical Field

[0001] The present invention relates to the technical field of multidimensional data analysis in big data, and in particular to an intelligent dynamic dimensional sorting method and system based on multidimensional feature parameters. Background Art

[0002] In the construction of multidimensional data models, the application data layer needs to generate rich dimension attributes when performing data analysis based on fact tables. When data analysts conduct actual business queries, the logical model is aggregated and returned based on the query indicators and dimension combinations, thereby providing strong support for multidimensional analysis of business data.

[0003] However, with the subdivision of specific business areas, the category characteristics of different business areas are becoming more and more differentiated.

[0004] Specifically, there are too many dimensions in a single business logic model, and there are more and more scenarios with personalized features of analysis categories. Some analysis dimensions, especially dimension groups with data belonging to the same dimension level but with very different business analysis perspectives, appear more and more frequently in the same analysis model. In addition, the capabilities of business analysts vary, and they have insufficient grasp of different business analysis scenarios, and cannot gain insight into truly valuable analysis scenarios. Under the goal of ensuring actual multidimensional analysis, if the dimension definition is subdivided based on all business scenarios, it will lead to the risk of excessive dimension widening, occupying too much physical storage space, and dimension explosion, which will also pose great challenges to dimension management.

[0005] For the above scenarios, currently, the following methods are generally used:

[0006] After acquiring the fact table data through collection, based on each piece of data in the fact table, the full dimension fields are matched using HIVE / HBASE dimension tables. By expanding the full dimension, the data analysis needs of different business parties for multi-dimensional decomposition can be met, such as Figure 1 shown.

[0007] However, in a single business logic model, there are more and more scenarios with a large number of different categories in the same dimensional level, strong personalized scenarios, and very different analysis perspectives.

[0008] For example, from a technical perspective, for the category dimensions derived from commodities, the personalized features need to be interpreted according to different category business analysis scenarios. The air-conditioning category needs to analyze the measurement data of the number of matches of commodities, and the alcoholic beverage category needs to analyze the degree, aroma, etc. If all dimensions are defined and managed separately at the dimension definition and management level, it will lead to problems such as excessive duplication of dimension definitions, doubling of physical storage space, and dimension explosion.

[0009] From a business perspective, due to the varying capabilities of data analysts, too many dimension types in the same analysis scenario will affect analysts' judgment, resulting in deviations in the direction of data analysis and an inability to gain insight into business changes. Summary of the invention

[0010] In view of the defects of the prior art, the purpose of the present invention is to provide an intelligent dynamic dimensional sorting method and system based on multi-dimensional feature parameters, so as to solve the problem that the multi-dimensional definition of business in the prior art cannot be controlled, reduce physical storage space and manual management costs through intelligent analysis and sorting, and help business data analysts to gain insight into key issues more quickly.

[0011] A first aspect of the present invention provides an intelligent dynamic dimensional sorting method based on multi-dimensional feature parameters, comprising:

[0012] Based on the standard dimension data of commodities stored in the data warehouse, create a commodity hierarchical dimension table containing the atomic dimensions of commodities and their hierarchical relationships;

[0013] According to the parameter type corresponding to each product, in combination with the hierarchical relationship table, a product parameter dimension table including all parameter types and corresponding parameter values ​​of the product is created;

[0014] Commodities at different levels are broken down according to parameter types, and each parameter value after the breakdown is sorted in a dimensionally differentiated manner using a sorting algorithm.

[0015] Furthermore, the method also includes: making the sorting results of all commodity parameters into a configurable BI report or intermediate table.

[0016] Furthermore, the decomposition of the parameter types includes: decomposing the parameter types separately according to different business scenarios or statistical dimensions to form a single decomposition table.

[0017] Furthermore, the sorting includes: forming a complete sorting report according to the hierarchical relationship, parameter type, calculation result and sorting sequence number of all the disassembly tables.

[0018] Furthermore, the sorting algorithm is one or a combination of variance comparison calculation method, mean method, median method, and normal distribution method.

[0019] Furthermore, the variance comparison calculation method includes:

[0020] The sample capacity of parameter values ​​under all parameter types is subjected to variance homogeneity check, and after passing the check, variance comparison calculation is performed. According to the variance calculation results, they are arranged in descending order according to the degree of dispersion. The variance calculation formula used is:

[0021] Among them, DX is the variance, EX is the overall mean, Xi is the variable, and Pi is the weight.

[0022] A second aspect of the present invention further provides an intelligent dynamic dimensional analysis system based on multi-dimensional feature parameters, the system comprising:

[0023] A commodity level dimension table creation module creates a commodity level dimension table containing the atomic dimensions of commodities and their hierarchical relationships according to the standard dimension data of commodities stored in the data warehouse;

[0024] A commodity parameter dimension table creation module creates a commodity parameter dimension table containing all parameter types and corresponding parameter values ​​of commodities according to the parameter types corresponding to each commodity and in combination with the hierarchical relationship table;

[0025] The intelligent sorting module breaks down commodities at different levels according to parameter types, and uses a sorting algorithm to perform dimensionally differentiated sorting on each parameter value after the breakdown.

[0026] Furthermore, the system also includes: a configuration module, which makes a configurable report model according to the sorting results of the intelligent sorting module.

[0027] Compared with the prior art, the intelligent dynamic dimensional sorting method and system based on multi-dimensional feature parameters provided by the present invention achieves the following technical effects:

[0028] 1. The present invention disassembles the products according to parameter types, and then sorts each calculated parameter value, skipping the BI report development process. It can make timely adjustments to the report structure for newly added or changed dimensions; in addition, data analysis and verification can be performed at any time through the smart configuration of the personalized report module.

[0029] 2. Since the dimensions of BI reports are flexible and changeable, there is no need to over-widen the fact table. Therefore, differentiated dimension sorting control can be performed for different business scenarios. The number of eigenvalues ​​of the fact table model can also be effectively controlled, which not only meets the analysis needs, but also solves the hidden dangers of over-widening of the analysis model and dimensional explosion to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is the formation process and example table of the commodity access result table in the background technology of the present invention.

[0031] Figure 2 It is a flowchart of an intelligent dynamic dimensional sorting method based on multi-dimensional feature parameters in an embodiment of the present invention.

[0032] Figure 3 This is an example diagram of a commodity level dimension table in an embodiment of the present invention.

[0033] Figure 4 This is an example diagram of a commodity parameter dimension table in an embodiment of the present invention.

[0034] Figure 5 This is an example diagram of a commodity parameter disassembly table in an embodiment of the present invention.

[0035] Figure 6 This is an example diagram of a commodity parameter sorting table in an embodiment of the present invention.

[0036] Figure 7 This is an example diagram of a report display in an embodiment of the present invention.

[0037] Figure 8 It is a schematic diagram of the architecture of an intelligent dynamic dimensional sorting system based on multi-dimensional feature parameters in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are preferred implementation methods of the present invention, and the description is for the purpose of illustrating the general principles of the present invention, and is not intended to limit the scope of the present invention. The scope of protection of the present invention shall be based on the claims, and all other embodiments obtained by ordinary technicians in this field without creative work based on the embodiments of the present invention shall fall within the scope of protection of the present invention.

[0039] Reference Figure 2 As shown, an embodiment of the present invention provides an intelligent dynamic dimensional sorting method based on multi-dimensional feature parameters, the method comprising:

[0040] Step S11: Create a commodity hierarchical dimension table including the atomic dimensions of the commodities and their hierarchical relationships according to the standard dimension data of the commodities stored in the data warehouse;

[0041] Mate Reference Figure 3 As shown, specifically, according to the commodity dimension table in the data warehouse, that is, the dictionary table storing standard dimension data, first define and create an atomic dimension, which is the primary key field of the dimension table, recording the code number and commodity name of the commodity, such as 11928436409 represents a certain model of Midea air conditioner, and 11398170651 represents a certain specification of Kweichow Moutai liquor; then, according to the commodity dimension relationship table in the data warehouse, that is, the relationship dimension table storing standard dimension data and its hierarchical category relationship, define the hierarchical relationship, such as 00001 represents the air conditioner category, 00002 represents the wine category, and 00003 represents the communication category. The atomic dimensions of all commodities and the corresponding category hierarchical relationships are combined to form a commodity hierarchical dimension table.

[0042] Step S12: Create a commodity parameter dimension table containing all parameter types and corresponding parameter values ​​of commodities according to the parameter type corresponding to each commodity and in combination with the hierarchical relationship table;

[0043] Mate Reference Figure 4 As shown, specifically, based on the commodity parameter table in the data warehouse, that is, the relational dimension table storing the characteristic parameters corresponding to the standard dimension data, and combined with the aforementioned hierarchical relationship table, a commodity parameter dimension table covering the parameter types and corresponding parameter values ​​of all analysis parameters of the commodity is created. The commodity parameter dimension table includes the commodity name (atomic dimension), commodity category (level), parameter type and corresponding parameter value. The commodity levels are sorted in sequence to form a parameter dimension table for all commodities.

[0044] Step S13: The commodities at different levels are decomposed according to the parameter types, and each parameter value after decomposition is sorted by dimension differentiation through a sorting algorithm.

[0045] In the data analysis of commodities, rich dimensional data represents multiple analysis perspectives. When faced with the decomposition of business problems, dimensions become the best tool for insight into data problems. The key to dimensional analysis is differential analysis. With the decomposition of dimensions, it is easier for data analysts to gain insight into data problems and fill in gaps.

[0046] To this end, the present invention, based on the parameter dimension table formed in step S12, decomposes the parameter types involved one by one according to the requirements of different application scenarios or statistical dimensions, and decomposes them into a decomposition table of each specific parameter type. Among them, each parameter type of a category contains multiple dimensions. For example, the air-conditioning category focuses on the horsepower, energy consumption, etc. Correspondingly, the parameter values ​​under the same parameter type can also be subdivided into different ones. For example, the parameter characteristic values ​​corresponding to the horsepower are 1 horsepower, 2 horsepower, 4 horsepower, etc. The parameter types under the wine category include degree, flavor, capacity, etc. The parameter values ​​under the degree also include 10 degrees, 30 degrees, 45 degrees, 53 degrees, etc. The parameter types under the communication category include memory, resolution, operating system, etc. Each parameter type can contain multiple different parameter values. Further decomposition of these parameter values ​​can find out more specific difference problems. Refer to Figure 5As shown in the figure, the decomposition results of 1 million GMV (transaction amount of product orders) of the air-conditioning category in different dimensions are shown in the figure. Among them, the left side is the GMV decomposition result under the horsepower dimension, for all parameter values ​​under the horsepower, and the right side is the GMV decomposition result under the energy consumption dimension. It can be seen from the figure that the GMV decomposition distribution under the horsepower dimension is relatively even, and the decomposition distribution under the energy consumption dimension is relatively discrete. Analysts can better evaluate the market size and user preferences of air conditioners under each energy consumption under the energy consumption dimension. For example, in the energy consumption dimension, the order amounts of 500W and 600W are the largest, which is conducive to decision makers to formulate personalized marketing strategies for products with different market values.

[0047] After all parameter types under each category can be split according to the above dimensions, many corresponding parameter value sample values ​​under a certain parameter type can be obtained. In order to obtain results with business value from a large number of parameter types, the present invention sorts the parameter value sample values ​​under different dimensions according to a sorting algorithm to obtain the business value ranking results under different parameter types. In order to facilitate intuitive display, all disassembly tables are sorted according to categories (categories are arranged according to hierarchical relationships), parameter types (parameter types in each category have multiple dimensions), calculation results (algorithm calculation results) and sorting numbers (corresponding rankings) to form a complete sorting report, such as Figure 6 As shown. In the sorting report, the ranking of the business value of commodities under different dimensions can be understood, and different dimensions under the same commodity have differentiated representation methods. As the business changes, the algorithm can make timely adjustments to the business. Whether adding categories, adding parameter types or adding parameter feature values, timely adjustments can be made through the sorting algorithm of the present invention.

[0048] Specifically, the sorting algorithm may be a variance comparison calculation method, a mean method, a median method, a normal distribution method, or the like.

[0049] Preferably, the sorting algorithm adopts variance comparison, taking variance comparison calculation as an example, including:

[0050] The sample capacity of parameter values ​​under all parameter types is subjected to variance homogeneity check, and after passing the check, variance comparison calculation and analysis are performed. According to the variance calculation results, they are arranged in descending order according to the degree of dispersion. The variance calculation formula used is:

[0051] Among them, DX is the variance, EX is the overall mean, Xi is the variable, and Pi is the weight. The variance can be used to find the degree of deviation of the random variable from the expected value, which can enable business personnel to find the dimensional value of a certain product and facilitate decision-making.

[0052] Based on the dimension table of commodity parameters obtained in step S12 and the ranking table of commodity parameters obtained in step S13, the parameter type and the parameter value range of the commodity are configured in the report through the report tool. If the data analyst has other adjustments to the ranking of the importance of the parameter type calculated in step S13, the tool can be used to make the parameter type a configurable model that gives priority to displaying the ranking results, such as Figure 7 As shown, only some of the top results are displayed, and the top rankings are more meaningful for data analysis. Since the dimensions of BI reports are flexible and do not need to be overly widened in the fact table, the number of characteristic values ​​or parameter values ​​in the fact table can be effectively controlled.

[0053] In addition, the sorting results in step S13 can also be made into a configuration intermediate table. The reporting tool can dynamically splice the parameter types into the front-end display interface by reading the intermediate table. Only the analysis results of the required dimensions can be selected to achieve flexible configuration and optimize resource utilization.

[0054] Corresponding to the above method embodiment, refer to Figure 8 As shown, another embodiment of the present invention further provides an intelligent dynamic dimensional analysis system based on multi-dimensional feature parameters, the system comprising: a commodity level dimension table creation module, a commodity parameter dimension table creation module, and an intelligent sorting module.

[0055] A commodity level dimension table creation module creates a commodity level dimension table containing the atomic dimensions and hierarchical relationships of commodities based on the standard dimension data of commodities stored in the data warehouse;

[0056] The commodity parameter dimension table creation module creates a commodity parameter dimension table containing all parameter types and corresponding parameter values ​​of commodities according to the parameter types corresponding to each commodity and in combination with the hierarchical dimension table;

[0057] The intelligent sorting module breaks down commodities at different levels according to parameter types, and uses a sorting algorithm to perform dimensionally differentiated sorting on each parameter value after the breakdown.

[0058] In order to provide business personnel with analysis of charts processed by the intelligent sorting module, the present invention also provides a configuration module. According to the sorting results of the intelligent sorting module, a model diagram that can display the sorting results is produced using the configuration tool in the report. The model diagram can display the results of multiple parameter types with high rankings. Alternatively, the sorting results can be produced into a configurable intermediate table. By reading the intermediate table, the report tool can select parameter data of any dimension for review.

[0059] In a single business logic model, different categories in the same dimension level have derived a large number of scenarios with strong personalized scenarios and very different analysis perspectives. In view of this, the present invention first divides all categories of goods according to categories, levels, parameter types and parameter values ​​through a commodity parameter dimension table creation module and a commodity parameter dimension table creation module, and creates dimensions covering all analysis parameters of the goods. When faced with business problem decomposition, dimensions become the best tool for insight into data problems. Therefore, all dimensions are decomposed according to the application scenarios to be analyzed. From the decomposed dimension table, the market size, user preferences, etc. of each segmented product under the dimension can be analyzed. In addition, according to the sorting algorithm, the goods under all parameter types are sorted according to different dimensions, and the parameter types with the highest rankings are set first. In this way, business personnel can intuitively understand the different rankings under different dimensions, which is convenient for business personnel to make accurate decisions.

[0060] The above description shows and describes several preferred embodiments of the present invention, but as mentioned above, it should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the invention concept described herein through the above teachings or the technology or knowledge of the relevant field. Changes and variations made by those skilled in the art do not depart from the spirit and scope of the present invention, and should be within the scope of protection of the claims attached to the present invention.

Claims

1. An intelligent dynamic dimension sorting method based on multi-dimensional feature parameters, characterized in that: The method comprises: According to the standard dimension data of the commodities stored in the data warehouse, a commodity hierarchical dimension table including the atomic dimensions and hierarchical relationships of the commodities is created. The standard dimension data includes a commodity dimension table and a commodity dimension relationship table. First, the atomic dimension is defined and created according to the commodity dimension table. The atomic dimension is the primary key field of the dimension table. Then, the hierarchical relationship is defined according to the commodity dimension relationship table. The atomic dimensions of the commodities and the corresponding category hierarchical relationships are combined to form the commodity hierarchical dimension table. According to the parameter type corresponding to each product, in combination with the hierarchical dimension table, a product parameter dimension table including all parameter types and corresponding parameter values ​​of the product is created; According to the commodity parameter dimension table, commodities at different levels are decomposed according to parameter types, and each parameter value after decomposition is sorted by dimension differentiation through a sorting algorithm, and the parameter type is made into a configurable model for displaying the sorting results first, and the sorting results are made into a configuration intermediate table, and the sorting results are controlled by a reporting tool to only display relevant data of the top-ranked parameter types; The decomposition of the parameter type includes: according to different business scenarios or statistical dimensions, the commodities at different levels are decomposed according to their parameter types to form a plurality of decomposition tables with a single parameter type, wherein the single parameter type includes a plurality of different parameter values.

2. The method according to claim 1, characterized in that The method further includes: making the ranking results of the commodity parameters into a configurable BI report or intermediate table.

3. The method according to claim 2, characterized in that The sorting includes: forming a complete sorting report according to the hierarchical relationship, parameter type, calculation result and sorting sequence number of all the disassembly tables.

4. The method according to claim 3, characterized in that The sorting algorithm is one or a combination of variance comparison calculation method, mean method, median method, and normal distribution method.

5. The method according to claim 4, characterized in that The variance comparison calculation method includes: The sample capacity of parameter values ​​under all parameter types is subjected to variance homogeneity check, and after passing the check, variance comparison calculation is performed. According to the variance calculation results, they are arranged in descending order according to the degree of dispersion. The variance calculation formula used is: Among them, DX is the variance, EX is the overall mean, Xi is the variable, and Pi is the weight.

6. The method according to claim 1, characterized in that The commodity level dimension table includes: the atomic dimension of the commodity and the category level; the commodity parameter dimension table includes the atomic dimension of the commodity, the category level, the parameter type, and the parameter value.

7. An intelligent dynamic dimensional sorting system based on multi-dimensional feature parameters, characterized in that: The system comprises: A commodity level dimension table creation module creates a commodity level dimension table containing the atomic dimensions and hierarchical relationships of commodities based on the standard dimension data of commodities stored in the data warehouse; A commodity parameter dimension table creation module, which creates a commodity parameter dimension table containing all parameter types and corresponding parameter values ​​of commodities according to the parameter types corresponding to each commodity and in combination with the hierarchical dimension table; An intelligent sorting module, which decomposes commodities at different levels according to parameter types based on the commodity parameter dimension table, and performs dimensionally differentiated sorting on each parameter value after decomposition through a sorting algorithm; The decomposition of the parameter type includes: the decomposition of the parameter type includes: according to different business scenarios or statistical dimensions, the commodities at different levels are decomposed according to their parameter types to form multiple decomposition tables with a single parameter type, and the single parameter type contains multiple different parameter values.

8. The system according to claim 7, characterized in that The system also includes: a configuration module, which makes a configurable report model according to the sorting results of the intelligent sorting module.

Images