Ffact table dimension reduction processing method and device, equipment and storage medium

Abstract

The invention relates to the technical field of data processing, and provides a fact table dimensionality reduction processing method and device, equipment and a storage medium, and the method comprises the steps: obtaining original fact table data; according to a preset dimension model, performing dimension reduction processing on the original fact table data to obtain corresponding target fact table data of multiple dimensions; and storing and displaying the target fact table data according to dimension classification. According to the technical scheme, the disk storage pressure is reduced, the data processing efficiency is improved, the situation that return logic occupies most resources is avoided, meanwhile, the situation that accurate data cannot be responded in time when a service department changes an analysis table sample is avoided, and the timeliness of service data is achieved.

Description

[Technical field]

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, equipment and medium for processing dimensionality reduction of a fact table. [Background technology]

[0002] In the construction of data marts, it is necessary to create various analytical reports and display graphics through the analysis scenarios of business departments, and intuitively reflect the business department's operating data, assessment data, analysis data, etc. through the display of reports and graphics. In the existing data mart construction and data display, the traditional use of fact tables is to process application data, and different reports and graphics use SQL statements to write the required result data through the fact table data. Fact tables are currently often at the granularity of sub-accounts. Although this design idea can meet the report query of various scenarios, there is a big drawback. Each report and graphic needs to extract data from a large fact table during the data processing process, which consumes more hardware resources, and both memory usage and CPU usage are at a high level.

[0003] like Figure 1 As shown, in the related technical solutions, the data of the display layer comes directly from the large wide table. Although there is no problem with the data source, there are disadvantages in efficiency and data volume.

[0004] In other words, the business departments currently need to obtain operating data, assessment data, analysis data, etc. through reports and cockpits. This part of the data is directly obtained through SQL queries on the fact table at the granularity of the account. This requires multiple queries and traversals on the fact table at the granularity of the account, which makes it impossible to release hardware resources in time, affecting the efficiency of the system. The storage of daily data in the fact table puts a lot of pressure on the disk space. [Summary of the invention]

[0005] The embodiments of the present application provide a method, device, equipment and medium for dimensionality reduction processing of a fact table, aiming to solve technical problems in related technologies such as a fact table occupying a large disk space, making it impossible to release hardware resources in time.

[0006] In a first aspect, an embodiment of the present application provides a method for dimensionality reduction processing of a fact table, comprising:

[0007] Get the original fact table data;

[0008] According to the preset dimensional model, the original fact table data is subjected to dimensionality reduction processing to obtain corresponding target fact table data of multiple dimensions;

[0009] The target fact table data is classified, stored and displayed according to the dimensions.

[0010] In one embodiment, optionally, the method further comprises:

[0011] Obtaining historical fact table data and historical query records corresponding to the historical fact table data;

[0012] Classify the historical query records according to different query dimensions to obtain classification results;

[0013] Count the number of historical query records corresponding to each type of query dimension, and determine the weight value corresponding to the type of query dimension according to the number of historical query records;

[0014] Compare the weight value corresponding to each type of query dimension with a preset threshold, and determine the query dimension with a weight value greater than the preset threshold as the target query dimension;

[0015] According to the target query dimension, a corresponding preset dimensional model is generated.

[0016] In one embodiment, optionally, generating a corresponding preset dimension model according to the target query dimension includes:

[0017] Determine dimension classification and dimension query index according to historical query records and corresponding query data of the target query dimension;

[0018] A corresponding preset dimensional model is generated according to the dimensional classification and dimensional query index.

[0019] In one embodiment, optionally, determining the weight value corresponding to the query dimension of this type according to the number of historical query records includes:

[0020] According to the number of historical query records of each type of query dimension, the confidence value corresponding to this type of query dimension in all historical query records is calculated according to the preset confidence calculation formula;

[0021] The confidence value is determined as the weight value corresponding to the query dimension of this type.

[0022] In one embodiment, optionally, the preset dimensional model includes at least one of the following: a customer dimensional model, a channel dimensional model, an organization dimensional model, a product dimensional model, and an organization-product dimensional model.

[0023] In one embodiment, optionally, according to a preset dimensional model, the original fact table data is subjected to dimensionality reduction processing to obtain corresponding target fact table data of multiple dimensions, including:

[0024] Determine the target dimension classification and target dimension query index corresponding to the preset dimension model;

[0025] Performing table structure and data structure analysis on the original fact table data to obtain analysis results;

[0026] According to the analysis result, target dimension data is extracted from the original fact table data according to the target dimension classification and target dimension query index;

[0027] Aggregate the target dimension data to obtain corresponding target fact table data.

[0028] In one embodiment, optionally, the method further comprises:

[0029] Receive input data query instructions;

[0030] Determine the target dimension classification corresponding to the data query instruction;

[0031] According to the target dimension classification, query and extract corresponding fact table data.

[0032] In a second aspect, an embodiment of the present application provides a fact table dimensionality reduction processing device, comprising:

[0033] The acquisition module is used to obtain the original fact table data;

[0034] A processing module, used to perform dimensionality reduction processing on the original fact table data according to a preset dimensional model to obtain target fact table data of corresponding multiple dimensions;

[0035] The display module is used to classify, store and display the target fact table data according to the dimensions.

[0036] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned fact table dimensionality reduction processing method when executing the computer program.

[0037] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned fact table dimensionality reduction processing method are implemented.

[0038] In the scheme implemented by the above fact table dimensionality reduction processing method, device, system, equipment and storage medium, the original fact table data is obtained; according to the preset dimensional model, the original fact table data is subjected to dimensionality reduction processing to obtain the corresponding target fact table data of multiple dimensions; according to the dimensional classification, the target fact table data is stored and displayed. In the present invention, the fact table data is subjected to dimensionality reduction processing, so that when performing subsequent operations such as data query, the target fact table data after dimensionality reduction can be directly queried, thereby reducing the disk storage pressure, improving data processing efficiency, avoiding the back-running logic from occupying most of the resources, and avoiding the failure to respond to accurate data in time when the business department changes the analysis table sample, thereby solving the timeliness of business data.

Brief Description of the Drawings

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A flow chart of a method for processing a fact table in related technology is shown.

[0041] Figure 2 A schematic flow chart of a method for dimensionality reduction of a fact table according to an embodiment of the present application is shown.

[0042] Figure 3 A schematic flowchart of step S202 in a fact table dimensionality reduction processing method according to an embodiment of the present application is shown.

[0043] Figure 4 A schematic flowchart of a method for dimensionality reduction of a fact table according to another embodiment of the present application is shown.

[0044] Figure 5 A schematic flowchart of a method for dimensionality reduction of a fact table according to yet another embodiment of the present application is shown.

[0045] Figure 6 A block diagram of a fact table dimensionality reduction processing device according to an embodiment of the present application is shown.

[0046] Figure 7 A schematic structural diagram of a computer device according to an embodiment of the present application is shown.

[0047] Figure 8 Another structural schematic diagram of a computer device according to an embodiment of the present application is shown. [Specific implementation method]

[0048] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0049] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0050] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0051] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0052] See also Figure 2 , Figure 2 A schematic flow chart of a method for reducing the dimension of a fact table according to an embodiment of the present application is shown. The method for reducing the dimension of a fact table is used to solve technical problems in related technologies such as a fact table occupying a large disk space, making it impossible to release hardware resources in a timely manner.

[0053] like Figure 2 As shown, according to an embodiment of the present application, a method for reducing the dimension of a fact table includes:

[0054] Step S201, obtaining original fact table data;

[0055] The following takes a specific original fact table data as an example to explain the technical solution in detail.

[0056] The original fact table data is shown in Table 1.

[0057] Table 1

[0058]

[0059]

[0060]

[0061] The fact table can also be called a financial instrument summary table. This table stores data at the level of individual accounts. For small and micro customers, the data volume of this table is at the level of hundreds of thousands, for medium-sized customers, the data volume of this table is at the level of millions to tens of millions, and for large customers, the data volume of this table is at the level of hundreds of millions. The data volume of this table for super large customers is even larger because this table summarizes the individual account data of each business. For example, for bank customers, this table includes corporate deposits, retail deposits, corporate loans, retail loans, bills, intermediary commission income, credit card business and other businesses. For Internet customers, this table may contain communication-level data.

[0062] Step S202, performing dimensionality reduction processing on the original fact table data according to a preset dimensional model to obtain corresponding target fact table data of multiple dimensions;

[0063] like Figure 3 As shown, in one embodiment, optionally, step S202 includes:

[0064] Step S301, determining the target dimension classification and target dimension query index corresponding to the preset dimension model;

[0065] In one embodiment, optionally, the preset dimensional model includes at least one of the following: a customer dimensional model, a channel dimensional model, an organization dimensional model, a product dimensional model, and an organization-product dimensional model.

[0066] Among them, the target dimension classification includes classification according to customer dimension, channel dimension, institution dimension, product dimension and institution product classification, etc. The target dimension query index includes the corresponding query index data under its dimension classification, such as interest income, capital cost, net interest income, fee income, income tax expense, profit before provision, economic capital cost, account balance and average account balance.

[0067] Step S302, performing table structure and data structure analysis on the original fact table data to obtain analysis results;

[0068] Step S303, extracting target dimension data from the original fact table data according to the analysis result and the target dimension classification and target dimension query index;

[0069] In this embodiment, target dimension data can be extracted from the original fact table data according to the target dimension classification and the target dimension query index. For example, for the customer dimension classification, the target dimension query index includes operating indicators, including interest income, capital cost, net interest income, fee income, income tax expense, pre-provision profit, economic capital cost, account balance and average account balance. Then, interest income, capital cost, net interest income, fee income, income tax expense, pre-provision profit, economic capital cost, account balance and average account balance can be extracted from the fact table data according to the customer number and customer name.

[0070] Step S304: Aggregate the target dimension data to obtain corresponding target fact table data.

[0071] Step S203: classify, store and display the target fact table data according to the dimensions.

[0072] For each dimensional classification, the corresponding data can be stored in the corresponding location. In this way, different dimensional classifications have corresponding storage locations, which makes it easy to obtain data from the corresponding storage locations according to the dimensional classification.

[0073] The following uses the customer dimension model, channel dimension model, organization dimension model, product dimension model and organization product dimension model as examples to illustrate the processing process of the original fact table data described in Table 1.

[0074] 1. Customer Dimension Model

[0075] 1. Based on the business plan, determine the indicators that need to be analyzed according to the customer dimension and those that need to be analyzed according to the customer dimension;

[0076] 2. Analyze the data table structure and data structure of the original fact table;

[0077] 3. Use technical language to aggregate business indicators based on customer information and produce customer dimension tables for subsequent customer analysis;

[0078] 4. Aggregation operations can be implemented in different ways such as Java and stored procedures; the core aggregation statements are as follows:

[0079] Select

[0080] Customer number,

[0081] Customer Name,

[0082] sum(interest income),

[0083] sum(capital cost),

[0084] sum(capital cost),

[0085] sum(net interest income),

[0086] sum(commission income),

[0087] sum(income tax expense),

[0088] sum(profit before provision),

[0089] sum(economic capital cost),

[0090] sum(account balance),

[0091] sum(average account balance)

[0092] From the original fact table data

[0093] group by customer number,

[0094] Customer name;

[0095] 5. Generate aggregation results and save them in the customer dimension table.

[0096] The customer dimension aggregation results are shown in Table 2.

[0097] Table 2

[0098]

[0099]

[0100] Through the above technical solution, the following technical effects are achieved: 1) Storage efficiency is improved. It is no longer necessary to store the fact table for the whole year (calculated as one year). Only the data of the customer dimension table for the whole year needs to be stored. The data volume of a single table is reduced (rows and columns are greatly reduced), which reduces the storage pressure of the hard disk; 2) When using the customer front-end report, it is no longer necessary to query the fact table with a large amount of data. Only the corresponding customer dimension table needs to be checked, which improves the query speed and enhances the customer system experience. 3) At the system function level, the query memory usage is reduced, so that multiple functions can be used with limited memory space.

[0101] 2. Product Dimension Model

[0102] 1. Based on the business plan, determine the indicators that need to be analyzed according to the product dimension and those that need to be analyzed according to the product dimension;

[0103] 2. Analyze the fact table structure and data structure;

[0104] 3. Aggregate business indicators based on customer information through technical language for subsequent analysis;

[0105] 4. Aggregation operations can be implemented in different ways such as Java and stored procedures; the core aggregation statements are as follows:

[0106] Select

[0107] Customer number,

[0108] Customer Name,

[0109] sum(interest income),

[0110] sum(capital cost),

[0111] sum(capital cost),

[0112] sum(net interest income),

[0113] sum(commission income),

[0114] sum(income tax expense),

[0115] sum(profit before provision),

[0116] sum(economic capital cost),

[0117] sum(account balance),

[0118] sum(average account balance)

[0119] From fact table

[0120] group by product number,

[0121] Product name;

[0122] 5. Generate aggregation results and save them in the product dimension table.

[0123] The aggregation results of product dimension are shown in Table 3.

[0124] Table 3

[0125] Product dimension table Product Code 2001001 Product dimension table Product Name Personal Loans Product dimension table Interest income 1000 Product dimension table Capital Cost 400 Product dimension table Capital Cost 200 Product dimension table Net interest income 600 Product dimension table Commission income 100 Product dimension table Income tax expense 10 Product dimension table Profit before provision 600 Product dimension table Economic capital cost 80 Product dimension table Account Balance 1000000 Product dimension table Average account balance 1000000

[0126] Through the above technical solution, the following technical effects are achieved: 1) Storage efficiency is improved. It is no longer necessary to store the fact table for the whole year (calculated as one year). Only the data of the product dimension table for the whole year needs to be stored. The data volume of a single table is reduced (the number of rows and columns is greatly reduced), which reduces the storage pressure of the hard disk; 2) When using the customer front-end report, it is no longer necessary to query the fact table with a large amount of data. Only the corresponding product dimension table needs to be checked, which improves the query speed and enhances the customer's system experience. 3) At the system function level, the query memory usage is reduced, so that multiple functions can be used with limited memory space.

[0127] 3. Channel Dimension Model

[0128] 1. Based on the business plan, determine the indicators that need to be analyzed according to the channel dimension and the indicators that need to be analyzed according to the channel dimension;

[0129] 2. Analyze the fact table structure and data structure;

[0130] 3. Aggregate business indicators based on customer information through technical language for subsequent analysis;

[0131] 4. Aggregation operations can be implemented in different ways such as Java and stored procedures; the core aggregation statements are as follows:

[0132] Select

[0133] Customer number,

[0134] Customer Name,

[0135] sum(interest income),

[0136] sum(capital cost),

[0137] sum(capital cost),

[0138] sum(net interest income),

[0139] sum(commission income),

[0140] sum(income tax expense),

[0141] sum(profit before provision),

[0142] sum(economic capital cost),

[0143] sum(account balance),

[0144] sum(average account balance)

[0145] From fact table

[0146] group by channel number,

[0147] Channel name.

[0148] 5. Generate aggregation results and save them in the channel dimension table.

[0149] The aggregation results of the channel dimension are shown in Table 4.

[0150] Table 4

[0151] Channel dimension table Channel Number 1001 Channel dimension table Channel Name Electronic Banking Channel dimension table Interest income 1000 Channel dimension table Capital Cost 400 Channel dimension table Capital Cost 200 Channel dimension table Net interest income 600 Channel dimension table Commission income 100 Channel dimension table Income tax expense 10 Channel dimension table Profit before provision 600 Channel dimension table Economic capital cost 80 Channel dimension table Account Balance 1000000 Channel dimension table Average account balance 1000000

[0152] Through the above technical solution, the following technical effects are achieved: 1) Storage efficiency is improved. It is no longer necessary to store the fact table for the whole year (calculated as one year). Only the data of the channel dimension table for the whole year needs to be stored. The data volume of a single table is reduced (the number of rows and columns is greatly reduced), which reduces the storage pressure of the hard disk; 2) When using the customer front-end report, it is no longer necessary to query the fact table with a large amount of data. Only the corresponding channel dimension table needs to be checked, which improves the query speed and enhances the customer's system experience. 3) At the system function level, the query memory usage is reduced, so that multiple functions can be used with limited memory space.

[0153] 4. Institutional Dimension Model

[0154] 1. Based on the business plan, determine the indicators that need to be analyzed according to the organizational dimension and those that need to be analyzed according to the organizational dimension;

[0155] 2. Analyze the fact table structure and data structure;

[0156] 3. Aggregate business indicators based on institutional information through technical language for subsequent analysis;

[0157] 4. Aggregation operations can be implemented in different ways such as Java and stored procedures; the core aggregation statements are as follows:

[0158] Select

[0159] Customer number,

[0160] Customer Name,

[0161] sum(interest income),

[0162] sum(capital cost),

[0163] sum(capital cost),

[0164] sum(net interest income),

[0165] sum(commission income),

[0166] sum(income tax expense),

[0167] sum(profit before provision),

[0168] sum(economic capital cost),

[0169] sum(account balance),

[0170] sum(average account balance)

[0171] From fact table

[0172] group by institution number,

[0173] Institution Name

[0174] 5. Generate aggregation results and save them in the organization dimension table;

[0175] The aggregation results of the institutional dimension are shown in Table 5.

[0176] Table 5

[0177]

[0178]

[0179] Through the above technical solution, the following technical effects are achieved: 1) Storage efficiency is improved. It is no longer necessary to store the fact table for the whole year (calculated as one year). Only the data of the channel dimension table for the whole year needs to be stored. The data volume of a single table is reduced (the number of rows and columns is greatly reduced), which reduces the storage pressure of the hard disk; 2) When using the customer front-end report, it is no longer necessary to query the fact table with a large amount of data. Only the corresponding channel dimension table needs to be checked, which improves the query speed and enhances the customer's system experience. 3) At the system function level, the query memory usage is reduced, so that multiple functions can be used with limited memory space.

[0180] V. Institutional Product Dimension Model

[0181] 1. Based on the business plan, determine the indicators that need to be analyzed according to the institutional product dimension and those that need to be analyzed according to the institutional product dimension;

[0182] 2. Analyze the fact table structure and data structure;

[0183] 3. Aggregate business indicators based on institutional product information through technical language for subsequent analysis;

[0184] 4. Aggregation operations can be implemented in different ways such as Java and stored procedures; the core aggregation statements are as follows:

[0185] Select

[0186] Customer number,

[0187] Customer Name,

[0188] sum(interest income),

[0189] sum(capital cost),

[0190] sum(capital cost),

[0191] sum(net interest income),

[0192] sum(commission income),

[0193] sum(income tax expense),

[0194] sum(profit before provision),

[0195] sum(economic capital cost),

[0196] sum(account balance),

[0197] sum(average account balance)

[0198] From fact table

[0199] group by institution, product number,

[0200] Organization and product name;

[0201] 5. Generate aggregation results and save them in the organizational product dimension table;

[0202] The aggregation results of the institutional product dimension are shown in Table 6.

[0203] Table 6

[0204] Institutional product dimension table Institution Number 100901 Institutional product dimension table Institution Name Branch Institutional product dimension table Product Code 2001001 Institutional product dimension table Product Name Personal Loans Institutional product dimension table Interest income 1000 Institutional product dimension table Capital Cost 400 Institutional product dimension table Capital Cost 200 Institutional product dimension table Net interest income 600 Institutional product dimension table Commission income 100 Institutional product dimension table Income tax expense 10 Institutional product dimension table Profit before provision 600 Institutional product dimension table Economic capital cost 80 Institutional product dimension table Account Balance 1000000 Institutional Product Dimension Table Average account balance 1000000

[0205] Through the above technical solution, the following technical effects are achieved: 1) Storage efficiency is improved. It is no longer necessary to store the fact table for the whole year (calculated on a one-year basis). Only the data of the institutional product dimension table for the whole year needs to be stored. The data volume of a single table is reduced (the number of rows and columns is greatly reduced), which reduces the storage pressure of the hard disk; 2) When using the customer's front-end report, it is no longer necessary to query the fact table with a large amount of data. It is only necessary to check the institutional product dimension table accordingly, which improves the query speed and increases the customer's system experience; 3) At the system function level, the query memory usage is reduced, so that multiple functions can be used within limited memory space.

[0206] like Figure 4 As shown, in one embodiment, optionally, the method further includes:

[0207] Step S401, obtaining historical fact table data and historical query records corresponding to the historical fact table data;

[0208] Step S402, classifying the historical query records according to different query dimensions to obtain classification results;

[0209] In this embodiment, the historical query records may be classified according to different query dimensions, such as classification according to customer dimension information, product dimension information, channel dimension information, organization dimension information, and organization product dimension information.

[0210] Step S403, counting the number of historical query records corresponding to each type of query dimension, and determining the weight value corresponding to the type of query dimension according to the number of historical query records;

[0211] In one embodiment, optionally, step S403 includes:

[0212] According to the number of historical query records of each type of query dimension, the confidence value corresponding to this type of query dimension in all historical query records is calculated according to the preset confidence calculation formula;

[0213] The confidence value is determined as the weight value corresponding to the query dimension of this type.

[0214] In this embodiment, the confidence value corresponding to each type of query dimension in all historical query records can be calculated according to the number of historical query records of each type of query dimension according to a preset confidence calculation formula, and the confidence is used as the weight value corresponding to this type of query dimension. The weight value corresponding to each type of query dimension is calculated by this method.

[0215] Step S404, comparing the weight value corresponding to each type of query dimension with a preset threshold, and determining the query dimension with a weight value greater than the preset threshold as the target query dimension;

[0216] In this embodiment, a preset threshold value, such as 0.6, can be set according to the needs, and the query dimension with a weight value greater than 0.6 is determined as the target query dimension. In this way, the corresponding preset dimension model can be constructed for the query dimension that is usually queried more frequently, so as to facilitate the query of similar data in the later stage.

[0217] Step S405: Generate a corresponding preset dimensional model according to the target query dimension.

[0218] In one embodiment, optionally, step S405 includes:

[0219] Determine dimension classification and dimension query index according to historical query records and corresponding query data of the target query dimension;

[0220] A corresponding preset dimensional model is generated according to the dimensional classification and dimensional query index.

[0221] In this embodiment, a preset dimensional model can be generated based on dimensional classifications and dimensional query indicators, wherein the dimensional classifications include customer dimensional classifications, product dimensional classifications, channel dimensional classifications, organization dimensional classifications, and organization product dimensional classifications, and the dimensional query indicators include business indicators.

[0222] like Figure 5 As shown, in one embodiment, optionally, the method further includes:

[0223] Step S501, receiving an input data query instruction;

[0224] Step S502, determining the target dimension classification corresponding to the data query instruction;

[0225] Step S503: query and extract corresponding fact table data according to the target dimension classification.

[0226] In this embodiment, when using the customer front-end report, it is no longer necessary to query the fact table with a large amount of data, but only to query the corresponding dimension classification table, which improves the query speed and enhances the customer system experience. At the same time, the query memory usage is reduced at the system function level, so that multiple functions can be used in limited memory space.

[0227] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0228] Figure 6 A block diagram of a fact table dimensionality reduction processing device according to an embodiment of the present application is shown.

[0229] like Figure 6 As shown, in a second aspect, an embodiment of the present application provides a fact table dimensionality reduction processing device 60, comprising:

[0230] An acquisition module 61 is used to acquire original fact table data;

[0231] A processing module 62 is used to perform dimensionality reduction processing on the original fact table data according to a preset dimensional model to obtain target fact table data of corresponding multiple dimensions;

[0232] The display module 63 is used to classify, store and display the target fact table data according to the dimensions.

[0233] In one embodiment, optionally, the device further comprises:

[0234] A record acquisition module, used to acquire historical fact table data and historical query records corresponding to the historical fact table data;

[0235] A classification module, used to classify the historical query records according to different query dimensions to obtain classification results;

[0236] A statistical module is used to count the number of historical query records corresponding to each type of query dimension, and determine the weight value corresponding to the type of query dimension according to the number of historical query records;

[0237] A comparison module, used to compare the weight value corresponding to each type of query dimension with a preset threshold, and determine the query dimension with a weight value greater than the preset threshold as the target query dimension;

[0238] A generation module is used to generate a corresponding preset dimension model according to the target query dimension.

[0239] In one embodiment, optionally, the generating module includes:

[0240] A first determining unit, configured to determine a dimension classification and a dimension query index according to the historical query records and corresponding query data of the target query dimension;

[0241] The model generation unit is used to generate a corresponding preset dimensional model according to the dimensional classification and dimensional query index.

[0242] In one embodiment, optionally, the statistics module includes:

[0243] A calculation unit, used to calculate the confidence value corresponding to each type of query dimension in all historical query records according to the number of historical query records of each type of query dimension and a preset confidence calculation formula;

[0244] A weight determination unit is used to determine the confidence value as a weight value corresponding to the query dimension of this type.

[0245] In one embodiment, optionally, the preset dimensional model includes at least one of the following: a customer dimensional model, a channel dimensional model, an organization dimensional model, a product dimensional model, and an organization-product dimensional model.

[0246] In one embodiment, optionally, the processing module includes:

[0247] A second determining unit, used to determine a target dimension classification and a target dimension query index corresponding to the preset dimension model;

[0248] An analysis unit, used to analyze the table structure and data structure of the original fact table data to obtain an analysis result;

[0249] An extraction unit, configured to extract target dimension data from the original fact table data according to the analysis result, the target dimension classification and the target dimension query index;

[0250] The aggregation unit is used to perform aggregation processing on the target dimension data to obtain corresponding target fact table data.

[0251] In one embodiment, optionally, the device further comprises:

[0252] A receiving module, used for receiving an input data query instruction;

[0253] A determination module, used to determine the target dimension classification corresponding to the data query instruction;

[0254] The query module is used to perform query according to the target dimension classification and extract the corresponding fact table data.

[0255] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned fact table dimensionality reduction processing method when executing the computer program.

[0256] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned fact table dimensionality reduction processing method are implemented.

[0257] For the specific definition of the fact table dimensionality reduction processing device, please refer to the definition of the fact table dimensionality reduction processing method above, which will not be repeated here. Each module in the above-mentioned fact table dimensionality reduction processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0258] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a service-side method for dimensionality reduction of a fact table.

[0259] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a client side of a fact table dimensionality reduction processing method.

[0260] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0261] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the method described in the first aspect of the embodiment is implemented when the processor executes the computer program.

[0262] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or electronic device can refer to the relevant description in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0263] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0264] It should be understood that, although the terms first, second, etc. may be used to describe the setting unit in the embodiments of the present application, these setting units should not be limited to these terms. These terms are only used to distinguish the setting units from each other. For example, without departing from the scope of the embodiments of the present application, the first setting unit may also be referred to as the second setting unit, and similarly, the second setting unit may also be referred to as the first setting unit.

[0265] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0266] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0267] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0268] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0269] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for dimensionality reduction of a fact table, characterized in that: include: Get the original fact table data; According to the preset dimensional model, the original fact table data is subjected to dimensionality reduction processing to obtain corresponding target fact table data of multiple dimensions; The target fact table data is classified, stored and displayed according to the dimensions.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining historical fact table data and historical query records corresponding to the historical fact table data; Classify the historical query records according to different query dimensions to obtain classification results; Count the number of historical query records corresponding to each type of query dimension, and determine the weight value corresponding to the type of query dimension according to the number of historical query records; Compare the weight value corresponding to each type of query dimension with a preset threshold, and determine the query dimension with a weight value greater than the preset threshold as the target query dimension; According to the target query dimension, a corresponding preset dimensional model is generated.

3. The method according to claim 2, characterized in that The generating a corresponding preset dimension model according to the target query dimension includes: Determine dimension classification and dimension query index according to historical query records and corresponding query data of the target query dimension; A corresponding preset dimensional model is generated according to the dimensional classification and dimensional query index.

4. The method according to claim 2, characterized in that: Determining the weight value corresponding to the query dimension of this type according to the number of historical query records includes: According to the number of historical query records of each type of query dimension, the confidence value corresponding to this type of query dimension in all historical query records is calculated according to the preset confidence calculation formula; The confidence value is determined as the weight value corresponding to the query dimension of this type.

5. The method according to claim 1, characterized in that The preset dimensional model includes at least one of the following: a customer dimensional model, a channel dimensional model, an organization dimensional model, a product dimensional model and an organization product dimensional model.

6. The method according to claim 1, characterized in that According to the preset dimensional model, the original fact table data is subjected to dimensionality reduction processing to obtain corresponding target fact table data of multiple dimensions, including: Determine the target dimension classification and target dimension query index corresponding to the preset dimension model; Performing table structure and data structure analysis on the original fact table data to obtain analysis results; According to the analysis result, target dimension data is extracted from the original fact table data according to the target dimension classification and target dimension query index; Aggregate the target dimension data to obtain corresponding target fact table data.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Receive input data query instructions; Determine the target dimension classification corresponding to the data query instruction; According to the target dimension classification, query and extract corresponding fact table data.

8. A fact table dimension reduction processing device, characterized in that: The device comprises: The acquisition module is used to obtain the original fact table data; A processing module, used to perform dimensionality reduction processing on the original fact table data according to a preset dimensional model to obtain target fact table data of corresponding multiple dimensions; The display module is used to classify, store and display the target fact table data according to the dimensions.

9. A computer device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, wherein the instructions are configured to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Computer executable instructions are stored, and the computer executable instructions are used to execute the method according to any one of claims 1 to 7.

Images