Index data analysis method, device, equipment and program product

Abstract

The invention discloses an index data analysis method and device, equipment and a program product, and relates to the technical field of data processing, and the method comprises the steps: dividing index data into different hierarchies based on a preset layering strategy, and obtaining a hierarchy label of the index data; obtaining business scene characteristics of the index data, performing scene classification on the index data in combination with a preset index coding rule, and performing physical isolation on a storage result of the index data in a distributed column database physical table of the scene classification based on the hierarchical label; and selecting target index data and data display parameters based on a user interface, and querying the target index data in the physically isolated storage result to generate a real-time index billboard. Through combination of layered architecture design, a scene-based storage strategy and dynamic interaction capability, data islands are eliminated, timeliness is improved, analysis capability is flexible, the requirement of a traditional BI tool for manually configuring a data set is cancelled, and query efficiency is remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to an index data analysis method and device, equipment and program product. BACKGROUND

[0002] With the advent of the big data era, data analysis plays a crucial role in many fields. Efficient processing and flexible analysis of data have become the key support for enterprise decision-making and business development. Data management and analysis technology is also evolving to meet the growing demand for diversification.

[0003] Currently, similar technologies have many limitations in actual application. On the one hand, data storage is in an island state, with each system storing data independently. Cross-system data sharing relies on the long chain process of extraction-transmission-loading (ETL), resulting in a serious lag in data timeliness, with an average delay of 4 hours or even longer, making it difficult to meet the real-time requirements of business scenarios. On the other hand, traditional BI tools have obvious shortcomings in generating and analyzing index dashboards. Manually configuring data sets is time-consuming and usually takes more than 30 minutes, and the refresh cycle is long, making it impossible to update index data in real time. In the drill-down analysis path, it relies on predefined levels and lacks flexibility, making it impossible to perform dynamic cross-dimension analysis, severely restricting the depth and breadth of data analysis.

[0004] These problems result in multiple contradictions in the actual application of existing technologies: data island phenomenon, manual configuration of data sets, and rigid drill-down analysis path. Therefore, there are problems of difficulty in integrating and utilizing data from different systems, poor timeliness, and rigid analysis capabilities.

[0005] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0006] The main purpose of the present application is to provide an index data analysis method, device, equipment and program product, which aims to solve the technical problems of difficulty in integrating and utilizing data from different systems, poor timeliness, and rigid analysis capabilities.

[0007] To achieve the above purpose, the present application provides an index data analysis method, which comprises:

[0008] dividing the index data into different levels based on a preset hierarchical strategy to obtain a hierarchical label of the index data;

[0009] obtain a business scene feature of the index data, perform scene classification on the index data in combination with a preset index coding rule, and perform physical isolation on a storage result of the index data in the scene classified distributed columnar database physical table based on the hierarchical label;

[0010] select target index data and data display parameters based on a user interface, query the target index data in the physically isolated storage result to generate a real-time index board.

[0011] In an embodiment, the different levels include an overall index level, a subdivided dimension index level, and a customer intensity data level; wherein the subdivided dimension index level includes at least one of a product dimension, a customer group dimension, and a time dimension, and each dimension supports hierarchical nesting and drilling down.

[0012] In an embodiment, the step of obtaining a business scene feature of the index data, and performing scene classification on the index data in combination with a preset index coding rule includes:

[0013] obtaining a business scene feature of the index data, and identifying a scene label of the index data according to the business scene feature;

[0014] generating a unique identifier based on a preset index coding rule in combination with the scene label and the hierarchical label; wherein the unique identifier includes at least one of a hierarchical code and a scene classification code;

[0015] storing index data with the same scene classification code in an independent distributed columnar database physical table based on the unique identifier.

[0016] In an embodiment, the step of performing physical isolation on a storage result of the index data in the scene classified distributed columnar database physical table based on the hierarchical label includes: in the distributed columnar database physical table, distributing index data with the same hierarchical code in the unique identifier to the same partition based on a partition key mechanism of the distributed columnar database.

[0017] In an embodiment, the step of obtaining a business scene feature of the index data, performing scene classification on the index data in combination with a preset index coding rule, and performing physical isolation on a storage result of the index data in the scene classified distributed columnar database physical table based on the hierarchical label further includes:

[0018] when a new business scene is added, identifying a scene label of index data of the new business scene, and associating a hierarchical label corresponding to the index data of the new business scene;

[0019] generate a unique identifier based on the scene label and the associated hierarchical label of the index data and a preset index coding rule; wherein the scene classification code in the unique identifier is a newly added scene classification code;

[0020] create an independent distributed columnar database physical table according to the newly added scene classification code, and divide the storage area of the independent distributed columnar database physical table through the partition key mechanism.

[0021] In an embodiment, the step of querying the target index data in the physically isolated storage result to generate a real-time index dashboard based on the user interface selected target index data and data display parameters includes:

[0022] selecting target index data through an online EXCEL interface to trigger a preset index coding rule analysis;

[0023] Based on the analysis result of the target index data, associate and match to the corresponding partition in the distributed columnar database physical table, and query the target index data in real time;

[0024] Set data display parameters through an online EXCEL interface to build an EXCEL template;

[0025] After the EXCEL template is built, fill in the target index data in real time and render to generate a real-time index dashboard.

[0026] In an embodiment, the step of querying the target index data in real time includes:

[0027] Obtain the hierarchical code and scene classification code in the analysis result of the target index data;

[0028] Trigger the pre-aggregation mechanism of the distributed storage engine to locate the target index data in the distributed columnar database physical table and the corresponding partition through the hierarchical code and scene classification code;

[0029] Based on the positioning information, extract the compressed target index data.

[0030] In an embodiment, the step of querying the target index data in the physically isolated storage result to generate a real-time index dashboard based on the user interface selected target index data and data display parameters includes:

[0031] According to the user-configured downlink link order, analyze the scene classification code and hierarchical code of the target index data in the downlink link;

[0032] Associate the target index data in the downlink link with the physical table through a preset coding-physical table mapping rule;

[0033] If the down-drilling link of the target sequence needs to be associated with physical tables of different scene classification codes, it is determined that the association is cross-scene, triggering an asynchronous extraction-transformation-loading (ETL) link to complete the missing target indicator data;

[0034] Otherwise, real-time query of the target indicator data generates the down-drilling link of the target sequence.

[0035] In addition, to achieve the above-mentioned purpose, the present application also provides an indicator data analysis device, which comprises:

[0036] An indicator layering module is configured to divide the indicator data into different levels based on a preset layering strategy to obtain a level label of the indicator data.

[0037] A storage management module is configured to obtain a business scene feature of the indicator data, perform scene classification on the indicator data in combination with a preset indicator coding rule, and physically isolate a storage result of the indicator data in a distributed columnar database physical table based on the level label.

[0038] A data display module is configured to query the target indicator data in the physically isolated storage result based on user interface selected target indicator data and data display parameters to generate a real-time indicator dashboard.

[0039] In addition, to achieve the above-mentioned purpose, the present application also provides an indicator data analysis device, which comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the above-mentioned indicator data analysis method.

[0040] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, which is executed by a processor to implement the steps of the above-mentioned indicator data analysis method.

[0041] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the above-mentioned indicator data analysis method.

[0042] The present application divides the indicator data into different levels based on a preset layering strategy to obtain a level label of the indicator data.

[0043] The business scene feature of the indicator data is obtained, and the scene classification of the indicator data is implemented in combination with a preset indicator coding rule, and the physical isolation is performed in a distributed columnar database physical table based on the level label.

[0044] Select target index data and data display parameters, query the target index data to generate real-time index board.

[0045] Or multiple technical solutions, at least have the following technical effects:

[0046] The present application divides the index data into different levels based on the preset hierarchical strategy, obtains the hierarchical label of the index data, acquires the business scene characteristics of the index data, combines the preset index coding rule to classify the scenes of the index data, and physically isolates the storage results of the index data in the distributed columnar database physical table based on the hierarchical label. Select target index data and data display parameters based on the user interface, query the target index data in the physically isolated storage results to generate real-time index board. The problem of data island is solved, the cross-scene data query efficiency is improved, the timeliness is improved; traditional BI tools need to predefine analysis model, cannot dynamically adapt to business demand; at the same time, it can freely configure the drilling path, customize real-time report, increase analysis ability and reduce maintenance cost. BRIEF DESCRIPTION OF DRAWINGS

[0047] The drawings incorporated into the specification and forming part of the specification, show the embodiments consistent with the present application, and together with the specification used to explain the principles of the present application.

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or prior art, the drawings needed in the embodiment or prior art description will be briefly introduced as follows, and obviously, other drawings can be obtained by those skilled in the art without creative labor.

[0049] Figure 1 The flowchart provided by the index data analysis method embodiment one of the present application;

[0050] Figure 2 The flowchart provided by the index data analysis method embodiment two of the present application;

[0051] Figure 3 The flowchart provided by the index data analysis method embodiment three of the present application;

[0052] Figure 4 The flowchart provided by the index data analysis method embodiment four of the present application;

[0053] Figure 5 The flowchart provided by the index data analysis method embodiment five of the present application;

[0054] Figure 6A flowchart provided for the index data analysis method of the sixth embodiment of the application is shown in the figure;

[0055] Figure 7 A brief flowchart of the index data analysis method provided for the seventh embodiment of the application is shown in the figure;

[0056] Figure 8 A brief flowchart of the index data analysis method provided for the eighth embodiment of the application is shown in the figure;

[0057] Figure 9 A brief flowchart of the index data analysis method provided for the ninth embodiment of the application is shown in the figure;

[0058] Figure 10 A module structure diagram of the index data analysis device of the embodiment of the application is shown in the figure;

[0059] Figure 11 A device structure diagram of the hardware running environment involved in the index data analysis method of the embodiment of the application is shown in the figure.

[0060] The object implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0061] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the application and do not limit the application.

[0062] In order to better understand the technical solutions of the application, the specific embodiments will be described in detail below with reference to the drawings and the specific embodiments.

[0063] The main solution of the embodiment of the application is:

[0064] Because the prior art is difficult to integrate and utilize data from different systems, it is poor in timeliness and rigid in analysis capability.

[0065] The application provides a solution, which divides index data into different levels based on a preset hierarchical strategy to obtain hierarchical labels of the index data; obtains business scenario features of the index data, classifies the index data in scenarios based on a preset index coding rule, and physically isolates storage results of the index data in a distributed columnar database physical table based on the hierarchical labels; selects target index data and data display parameters based on a user interface, and queries the target index data in the physically isolated storage results to generate a real-time index dashboard. The solution solves the problem of data islands, improves cross-scenario data query efficiency and timeliness, and can dynamically adapt to business requirements while predefining analysis models for traditional BI tools. At the same time, it can freely configure drill-down paths and customize real-time reports to increase analysis capability and reduce maintenance costs.

[0066] Based on this, the application embodiment provides an index data analysis method, referring to Figure 1 , Figure 1 The flowchart of the first embodiment of the index data analysis method of the application is shown in the figure.

[0067] In this embodiment, the index data analysis method comprises steps S10-S30:

[0068] Step S10, dividing the index data into different levels based on a preset hierarchical strategy to obtain the hierarchical label of the index data;

[0069] It should be noted that the preset hierarchical strategy refers to a hierarchical rule system defined in advance based on business logic and data characteristics, which is used to divide the index data into a hierarchical structure with logical association. The hierarchical label of the index data is essentially assigned to the unique level identifier of each index, which is used to mark its level. By constructing a hierarchical data model, multi-level drilling analysis from macro to micro is realized; at the same time, the hierarchical identifier is provided for scene-based storage, which optimizes the data retrieval efficiency; reduces the complexity of cross-scene or cross-level data association.

[0070] In a possible implementation, the preset hierarchical strategy can be divided into three layers based on the "pyramid model": top layer: overall index; middle layer: subdivided dimension index; bottom layer: customer granularity data.

[0071] Step S20, obtaining the business scene characteristics of the index data, combining the preset index coding rule to classify the index data by scene, and physically isolating the storage result of the index data in the distributed columnar database physical table based on the hierarchical label;

[0072] It should be noted that the business scene characteristics refer to the scene attribute of the index data in business operation; the preset index coding rule is a structured coding system designed to realize cross-system data interconnection; the distributed columnar database can use ClickHouse; the columnar columnar database physical table refers to a distributed data table constructed based on columnar storage engines such as ClickHouse, which significantly improves the index data retrieval performance by column compression storage characteristics; the physical isolation refers to storing the index data of different scene classifications in different physical tables independently, and further subdividing the storage area through partition keys to realize the alignment of storage structure and business logic. It can be seen that the same scene data is stored in a centralized manner, avoiding cross-table association overhead, improving query efficiency, and reducing data island phenomenon.

[0073] Step S30, based on the user interface, selecting target index data and data display parameters, querying the target index data in the physically isolated storage result to generate a real-time index dashboard.

[0074] It should be noted that the target indicator data refers to the set of indicators to be analyzed selected by the user through the interactive interface; the selection logic of the target indicator data may include scenario screening, hierarchical filtering, manual checking, dimension configuration, etc.

[0075] In one possible implementation, the real-time indicator dashboard generation process includes: first querying the route, locating the target physical table and its partitions according to the indicator code, then performing aggregation acceleration, that is, calling the ClickHouse pre-aggregation view to quickly obtain the results, and finally performing visual rendering to generate the real-time indicator dashboard.

[0076] To summarize, this application solves the problem of data silos, improves the efficiency of cross-scenario data queries, and improves timeliness. Traditional BI tools require predefined analysis models and cannot dynamically adapt to business needs. At the same time, they can freely configure drill-down paths and generate customized real-time reports, increasing analytical capabilities and reducing maintenance costs.

[0077] Furthermore, the different levels include an overall indicator layer, a segmented dimension indicator layer, and a customer strength data layer; wherein the segmented dimension indicator layer includes at least one of a product dimension, a customer group dimension, and a time dimension, and each dimension supports hierarchical nested drill-down.

[0078] It's important to note that hierarchical nested drilldown refers to the ability to perform multi-level penetration analysis within the same dimension. For example, within the product dimension, you can drill down from the home appliance category to the air conditioner subcategory, then from the air conditioner subcategory to a specific brand of inverter air conditioner, and finally to the inventory and sales figures for a specific inverter air conditioner model. During this process, the system extracts data layer by layer using the ClickHouse pre-aggregation mechanism, combined with the partitioning strategy of the distributed engine storage, to improve query efficiency.

[0079] Further, see Figure 2 The second embodiment of the index data analysis method of this application provides a flow chart based on the above Figure 2 The example diagram shown further refines the preparation steps of "obtaining the business scenario characteristics of the indicator data and classifying the indicator data by scenarios in combination with the preset indicator coding rules" in step S20, including steps A201 to A203:

[0080] Step A201: Acquire business scenario features of the indicator data, and identify scenario tags of the indicator data according to the business scenario features;

[0081] It should be noted that the business scenario feature refers to a set of attributes that the index data exhibits in a specific business environment; the core role of the business scenario feature is to strongly bind the index data with the specific business logic, ensuring that the analysis perspective is aligned with the business needs. The scenario label is an abstract identifier of the business scenario feature.

[0082] In a possible implementation, the identification of the scenario label can be achieved by, for example, rule matching based on a predefined rule library, machine learning by training a classification model to automatically identify the scenario label, and manual intervention allowing users to manually specify the scenario label when entering the index.

[0083] Step A202: Based on the preset index coding rule, the scenario label and the hierarchical label are combined to generate a unique identifier; wherein the unique identifier includes at least one of hierarchical coding and scenario classification code;

[0084] It should be noted that the hierarchical coding is used to mark the level of the index; the scenario classification code is used to identify the business scenario category; in addition, the unique identifier can also include data bloodline marks, which record the processing path, source system and version number of the index, and cross-scene traceability.

[0085] In a possible implementation, according to the index coding rule, the unique identifier is generated by combining the scenario label, the hierarchical label, the data version, and the data bloodline mark.

[0086] Step A203: Based on the unique identifier, the index data with the same scenario classification code is stored in an independent distributed columnar database physical table.

[0087] It should be noted that the distributed columnar database physical table at least includes a physical table constructed by a columnar storage engine such as ClickHouse, and its core advantages include: columnar compression, continuous storage of columns of the same data type, which can improve the compression rate by 3-5; pre-aggregation optimization, pre-computing aggregation views for high-frequency query indexes. Effectively improve the response time; partition, divide the storage area according to time or business unit, etc.

[0088] The system assigns the index data with the same scenario classification code in the identifier to the same ClickHouse physical table, and further divides the storage area using the partition key mechanism, pre-aggregates high-frequency indexes, etc. This storage method can effectively reduce the query response time of the index data to improve efficiency, or quickly locate the corresponding exclusive scenario index through the scenario label, avoid confusion with other scenario data, and achieve accurate analysis.

[0089] Furthermore, the physical isolation of the storage results of the indicator data in the distributed column database physical table of the scenario classification based on the hierarchical label includes: in the distributed column database physical table, based on the partition key mechanism of the distributed column database, allocating the indicator data with the same hierarchical encoding in the unique identifier to the same partition.

[0090] It should be noted that in this embodiment, the "level label" specifically refers to the level identifier defined by a preset layering strategy (e.g., overall indicator layer, detailed dimension layer, and customer-granular data layer). For example, "L1" represents strategic-level summary indicators, "L2" represents business line dimension aggregate indicators, and "L3" represents customer-granular detailed data. The purpose of the level label is to align business logic with the physical storage structure, ensuring that data drill-down links strictly match business analysis requirements.

[0091] The "partition key mechanism" is the core data management strategy of the distributed columnar database, which divides the data into independent storage areas by specifying partition keys (such as the time field date, business unit org_id). In this embodiment, the system generates partition keys based on the hierarchical code (such as "L1" and "L2"), and assigns the indicator data of the same level to the same physical partition. For example, the overall indicator layer (L1) data is stored in daily partitions, while the detailed dimension layer (L2) data is partitioned by business unit. Through this design, the target partition can be quickly located during query, reducing the resource consumption of full table scans. Through this technical solution, the partition key accurately locates the data block, and the query efficiency is improved; the hierarchical label is strongly bound to the partition rule, avoiding manual maintenance of the mapping relationship and reducing management complexity.

[0092] Further, refer to Figure 3 The third embodiment of the index data analysis method of this application provides a flow chart based on the above Figure 3 The example diagram shown further refines the preparation steps of "obtaining the business scenario characteristics of the indicator data, classifying the indicator data according to the preset indicator coding rules, and physically isolating the storage results of the indicator data in the distributed column database physical table of the scenario classification based on the hierarchical label" in step S20, including steps A301 to A303:

[0093] Step A301: When a new business scenario is added, the scenario tag of the indicator data of the new business scenario is identified, and the level tag corresponding to the indicator data of the new business scenario is associated;

[0094] It should be noted that the newly added business scenarios refer to specific analysis scenarios added during the process of business expansion or process optimization. In this embodiment, the processing flow of newly added business scenarios is designed to achieve rapid adaptation through dynamic encoding and storage expansion. It does not require the reconstruction of the existing data architecture, and the newly added scenarios can be seamlessly integrated. At the same time, independent physical tables are used to avoid cross-scenario data contamination.

[0095] Step A302: Based on the scenario tags and associated level tags of the indicator data, a unique identifier is generated according to a preset indicator coding rule; wherein the scenario classification code in the unique identifier is a newly added scenario classification code;

[0096] In one possible implementation, the identification of new scenarios can be achieved through rule mapping, machine learning assistance, or manual intervention.

[0097] Step A303: Create an independent distributed column-based database physical table according to the newly added scenario classification code, and divide the storage area of ​​the independent distributed column-based database physical table by using the partition key mechanism.

[0098] It should be noted that physical tables are created in ClickHouse based on the newly added scenario classification codes, and the partition key mechanism is used to divide storage areas. High-frequency indicators are pre-aggregated and compressed and stored as materialized views. When adding new scenarios, only the coding rules need to be expanded and the physical tables created, which is compatible with the existing architecture.

[0099] Further, refer to Figure 4 The fourth embodiment of the index data analysis method of this application provides a flow chart based on the above Figure 4 The example diagram further refines the preparation steps of "selecting target indicator data and data display parameters based on the user interface, and querying the target indicator data in the physically isolated storage results to generate a real-time indicator dashboard" in step S30, including steps A401 to A404:

[0100] Step A401: Select target indicator data through the online EXCEL interface to trigger the preset indicator coding rule analysis;

[0101] It should be noted that the online Excel interface is an Excel-like interactive interface implemented using web technology. It allows users to select indicators and design reports through operations such as dragging, selecting, and editing formulas. Its core function is to lower the user barrier to entry and accommodate complex Chinese reporting requirements (such as multi-level headers and merged cells). The coding rule parsing process uses the system's built-in rule engine to map the indicator name selected by the user to a unique identifier, which is then associated with the physical storage location.

[0102] Step A402: Based on the analysis result of the target indicator data, associate and match to the corresponding partition in the distributed columnar database physical table, and query the target indicator data in real time;

[0103] Step A403: Set data display parameters through an online EXCEL interface, and build an EXCEL template;

[0104] It should be noted that the data display operation includes but is not limited to: visualization types such as column chart, line chart, heat map, etc.; style configuration such as color theme, font size, dynamic label, etc.; interaction rules such as drill-down level depth, dimension cross-combination logic, etc.

[0105] Step A404: After the EXCEL template is built, fill in the target indicator data in real time and render to generate a real-time indicator dashboard.

[0106] It should be noted that by building a template and filling in data through an online EXCEL interface, the system can achieve the following goals, such as zero coding configuration, allowing business personnel to generate professional reports without SQL or programming skills; real-time guarantee, ClickHouse pre-aggregation and columnar storage supporting second-level data pulling to respond to user indicator queries; flexible configuration, template supporting reuse and nesting, adapting to multi-scenario analysis needs.

[0107] In a feasible implementation, a certain retail enterprise needs to monitor the sales performance of various categories during the “618 big promotion”:

[0108] The user selects the target indicators “home appliance category sales” and “cosmetic category conversion rate” through the online EXCEL interface, triggering the rule engine to analyze;

[0109] The system matches the coding rules according to the indicator name to generate a unique identifier MKT_L2_JD618 (scenario code = MKT, level = L2, scenario identifier = JD618).

[0110] Based on the identifier MKT_L2_JD618, locate the partition JD618 of the ClickHouse physical table mkt_sales_l2;

[0111] Call the pre-aggregation view to extract compressed daily granularity sales data, with a response time ≤ 800 ms.

[0112] The user designs a report template on the online EXCEL interface:

[0113] The header merged cell displays “618 big promotion sales report”;

[0114] The home appliance category uses a stacked column chart to display the brand proportion, and the cosmetic category uses a line chart to display the conversion rate trend;

[0115] Set drill down rules: click on the column chart to drill down to single product sales details (L3 level).

[0116] The system fills in query data in real time, renders to generate visual dashboards, and generates shareable URL links.

[0117] This embodiment shortens the dashboard generation time from 30 minutes of traditional BI tools to 3 seconds, helps to improve efficiency and quickly adjust strategies; supports dynamic adjustment of display parameters (such as switching to a map heat map), without the need for re-development, enhancing system flexibility; through the partition key mechanism, only the storage area corresponding to the identifier is scanned, reducing resource consumption.

[0118] Further, referring to Figure 5 , the real-time query of the target indicator data includes steps A501-A503:

[0119] Step A501; obtain the hierarchical encoding and scene classification code in the analysis result of the target indicator data;

[0120] Step A502: Trigger the pre-aggregation mechanism of the distributed storage engine, and locate the target indicator data in the distributed columnar database physical table and the corresponding partition through the hierarchical encoding and scene classification code;

[0121] Step A503: Extract the compressed target indicator data based on the positioning information.

[0122] It should be noted that the compressed target indicator data adopts a columnar storage format; when the pre-aggregation mechanism is triggered, the system quickly filters out the target data block through the hierarchical encoding and scene classification code, and directly reads the pre-aggregated compressed data, thereby compressing the query response time from minutes to milliseconds.

[0123] Further, referring to Figure 6 , the index data analysis method of the present application provides a flowchart, based on the above Figure 6 example, the step after "querying the target indicator data in the physically isolated storage result based on the user interface selected target indicator data and data display parameters to generate real-time indicator dashboards" in step S30 is further refined, including steps A601-A604:

[0124] Step A601: According to the user-configured drill down link order, analyze the scene classification code and hierarchical encoding of the target indicator data in the drill down link;

[0125] Step A602: Associate the target indicator data in the drill down link with the physical table through the preset encoding-physical table mapping rule;

[0126] Step A603: If the down-drill link of the target sequence needs to associate physical tables of different scene classification codes, it is determined as cross-scene association, triggering asynchronous extract-transform-load (ETL) link to complete missing target indicator data;

[0127] Step A604: Otherwise, real-time query of target indicator data generates the down-drill link of the target sequence.

[0128] It should be noted that the down-drill link refers to a hierarchical analysis path from macro indicators to micro data, and its core function is to decompose complex business problems into sub-problems that can be explored layer by layer; the purpose of user configuring the down-drill link is to freely combine analysis paths according to business needs, accelerate query response through pre-aggregation and partitioning strategies, and automatically complete missing data when cross-scene association, avoiding analysis link breakage.

[0129] In a possible implementation, cross-scene association determination and processing can be compared through data bloodline marking, analyzing the data bloodline marking of the target indicator, detecting whether there is cross-scene dependence; asynchronous ETL link, extracting data of the dependent scene on demand, writing to the current physical table after conversion according to the target scene format; and dynamic association query, temporarily associating different scene tables through a globally unique identifier, ensuring query logic consistency.

[0130] Among them, it is assumed that the user configures the link sequence as: promotion sales (scene code MKT, level L2) → inventory turnover rate (scene code INV, level L2) → single product inventory details (scene code INV, level L3); the system analyzes the codes MKT_L2 and INV_L2, and associates them to the tables mkt_l2_metrics and inv_l2_metrics through mapping rules; the promotion sales data (pre-aggregation view view_sales_agg) and the inventory turnover rate data (partition key partition_by_org) are extracted.

[0131] Cross-scene association processing is: when the association requirement of the promotion scene (MKT) and the inventory scene (INV) is detected, an asynchronous ETL link is triggered; single product inventory data is incrementally extracted from the inventory scene table inv_l2_metrics, and fields such as unified timestamp and organization code are converted according to the promotion scene coding rules; the converted data is written to the extended partition of the promotion scene table mkt_l2_metrics, and the bloodline marking library record is updated to complete the path.

[0132] In addition, the user can dynamically adjust the link sequence, and the system automatically redirects to the corresponding physical table.

[0133] The cross-scene correlation analysis period is greatly shortened in the embodiment, data is supplemented by asynchronous ETL, storage redundancy is reduced, and ETL development workload of cross-system data synchronization is reduced; the visual link configuration interface supports drag adjustment, and the operation threshold tends to be zero.

[0134] In a possible implementation, referring to Figure 7 , a user checks a target index in an "analysis board" interface, and the system parses the unique identifier of the index according to an index coding rule; when the user configures a drill-down path at the index board, the system matches a physical table partition through hierarchical coding (such as L1→L2→L3), triggers a ClickHouse pre-aggregation query; when the user combines multiple dimensions (such as time + region) at the index board, if cross-scene correlation is needed, the system triggers an asynchronous ETL link to supplement data; and for data query strategy selection, ClickHouse (CK) is used for real-time aggregation query (such as pre-computing sales trend), and TDSQL is used for transactional operation (such as saving board configuration metadata); finally, the board is saved, an exchangeable board (such as a heat map, a detail table, etc.) is generated through an online EXCEL interface, and URL sharing or embedding into a third-party system is supported.

[0135] In addition, a user can also check an index and an agency level (such as "headquarters→South China branch") through an EXCEL-like interface, the system automatically parses to corresponding hierarchical coding, and correlates a ClickHouse physical table partition; the user can perform Chinese-style report merged cells (such as table header merging to display "regional sales summary"), multi-level table headers (such as hierarchical display of "category→sub-category→single product"), and cross-column formulas (such as automatic mapping to a database field) on the online EXCEL; the user performs template uploading on the online EXCEL, predefines a template to bind a ClickHouse table, and fills data in real time; then, CK / TDSQL collaborative data query and board saving are performed, and a board is generated and saved.

[0136] In a possible implementation, referring to Figure 8 , for index analysis, a user checks a target index through an interactive interface, or arranges an index structure, or sets display configuration (such as agency type, default date, etc.); and for online EXCEL analysis, a user can check target indexes, dimensions, or agency parameters, and fill them into an EXCEL template, and finally generate an interactive report together with the index analysis.

[0137] In a possible implementation, referring to Figure 9The user selects a business scenario through an interactive interface, checks target indicators, and generates a dashboard (report). During the generation of the dashboard (report), for indicator analysis, different levels can be selected for drilling down, such as drilling down to a customer group under a product dimension, drilling down to a product under a customer group, and drilling down to a customer. The drilling down process does not affect the display of the original node. For online EXCEL analysis, multiple dimensions can be selected for combination. The final data display is the result of the combination of the dimensions, or both product and customer group are selected, and the final result of the combination of the product and customer group is displayed. Therefore, cross analysis of data of different dimensions can be intuitively observed.

[0138] It should be noted that the above examples are only used for understanding the present application and do not constitute a limitation on the index data analysis method of the present application. More forms of simple transformation based on this technical concept are within the protection scope of the present application.

[0139] The present application also provides an index data analysis device, which is described in detail as follows. Figure 10 The index data analysis device comprises:

[0140] 10, an index layering module, configured to divide index data into different levels based on a preset layering strategy, to obtain a level label of the index data;

[0141] 20, a storage management module, configured to obtain a business scenario feature of the index data, to perform scenario classification on the index data in combination with a preset index coding rule, and to physically isolate a storage result of the index data in a distributed columnar database physical table based on the level label;

[0142] 30, a data display module, configured to query target index data in the physically isolated storage result based on a user interface selected target index data and data display parameters, to generate a real-time index dashboard.

[0143] The index data analysis device provided by the present application adopts the index data analysis method in the above embodiments, and can solve the technical problems of difficulty in integrating and utilizing data from different systems, poor timeliness, and rigid analysis capability. Compared with the prior art, the beneficial effects of the index data analysis device provided by the present application are the same as those of the index data analysis method provided by the above embodiments, and other technical features in the index data analysis device are the same as those disclosed in the above embodiments, which will not be repeated here.

[0144] The application provides a kind of index data analysis equipment, index data analysis equipment includes: at least one processor;And, with at least one processor communication connection's memory;Wherein, memory stores the instruction that can be executed at least one processor, instruction is executed at least one processor, to enable at least one processor to execute the index data analysis method in above-mentioned embodiment one.

[0145] Reference is made below Figure 11 , it shows the structure diagram suitable for being used to realize the index data analysis equipment of the embodiment of the application.The index data analysis equipment in the embodiment of the application can include but not limited to mobile terminal such as mobile phone, notebook computer, digital broadcast receiver, PDA (Personal Digital Assistant: personal digital assistant), PAD (PortableApplication Description: tablet computer), PMP (Portable Media Player: portable multimedia player), vehicle terminal (such as vehicle navigation terminal) and the like and fixed terminal such as digital TV, desktop computer and the like. Figure 11 The index data analysis equipment shown is only an example, should not bring any limitation to the function and use range of the embodiment of the application.

[0146] As Figure 11 Shown, index data analysis equipment can include processing device 1001 (for example, central processing unit, graphics processor and the like), it can be executed according to the program stored in read-only memory 1002 or the program loaded from storage device 1003 to random access memory 1004 various appropriate actions and processing.In random access memory 1004, various programs and data required for index data analysis equipment operation are also stored.Processing device 1001, read-only memory 1002 and random access memory 1004 are connected to each other by bus 1005.Input / output interface 1006 is also connected to bus.Usually, the following systems can be connected to input / output interface 1006: input device 1007 including, for example, touch screen, touch pad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope and the like;Output device 1008 including, for example, liquid crystal display (LCD: Liquid Crystal Display), speaker, vibrator and the like;Including, for example, magnetic tape, hard disk and the like storage device 1003 and communication device 1009.Communication device 1009 can allow index data analysis equipment and other equipment to communicate wirelessly or wired to exchange data.Although the index data analysis equipment with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown.Alternatively, more or less systems can be implemented or provided.

[0147] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network through a communication device, or installed from the storage device 1003, or installed from the read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiments disclosed in the present application are executed.

[0148] The index data analysis device provided by the present application adopts the index data analysis method in the above-mentioned embodiments, and can solve the technical problems of difficulty in integrating and utilizing data from different systems, poor timeliness, and rigid analysis capability. Compared with the prior art, the index data analysis device provided by the present application has the same beneficial effects as the index data analysis method provided by the above-mentioned embodiments, and other technical features in the index data analysis device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0149] It should be understood that parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the description of the above-mentioned embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0150] The above is merely specific implementation of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0151] The present application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e. computer program) for executing the index data analysis method in the above-mentioned embodiments.

[0152] The computer readable storage medium provided in the application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system or device, or any combination of the above. More specific examples of the computer readable storage medium may include, but are not limited to, an electric connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiment, the computer readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), and the like, or any suitable combination of the above.

[0153] The computer readable storage medium described above may be contained in the index data analysis device, or may exist separately without being assembled into the index data analysis device.

[0154] The computer readable storage medium described above carries one or more programs, which, when executed by the index data analysis device, cause the index data analysis device to: divide index data into different levels based on a preset hierarchical strategy to obtain a hierarchical label of the index data; obtain a business scenario feature of the index data, perform scene classification on the index data in combination with a preset index coding rule, and physically isolate a storage result of the index data in the distributed columnar database physical table based on the hierarchical label; select target index data and data display parameters based on a user interface, query the target index data in the physically isolated storage result to generate a real-time index board.

[0155] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0156] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0157] The modules involved in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0158] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., a computer program) for executing the above-mentioned index data analysis method, and can solve the technical problems of difficulty in integrating and utilizing data from different systems, poor timeliness, and rigid analysis capability. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the index data analysis method provided by the above-mentioned embodiments, and will not be described here.

[0159] The application further provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the index data analysis method as described above.

[0160] The computer program product provided by the application can solve the technical problems of difficulty in integrating and utilizing data from different systems, poor timeliness, and rigid analysis capability. Compared with the prior art, the beneficial effects of the computer program product provided by the application are the same as those of the index data analysis method provided by the above-mentioned embodiments, and are not described here.

[0161] The above only describes some embodiments of the application, and does not limit the patent scope of the application. Any equivalent structural transformation, direct / indirect application in other related technical fields, or the like made by using the content of the application specification and drawings within the technical concept of the application is included in the patent protection scope of the application.

Claims

1. A method for analyzing index data, characterized in that: The indicator data analysis method includes: Dividing the indicator data into different levels based on a preset hierarchical strategy to obtain level labels for the indicator data; Obtaining business scenario features of the indicator data, classifying the indicator data by scenario in combination with preset indicator coding rules, and physically isolating the storage results of the indicator data in the physical table of the distributed column-based database of the scenario classification based on the hierarchical labels; Target indicator data and data display parameters are selected based on the user interface, and the target indicator data is queried in the storage results after physical isolation to generate a real-time indicator dashboard.

2. The indicator data analysis method according to claim 1, wherein: The different levels include an overall indicator layer, a segmented dimension indicator layer, and a customer strength data layer; wherein the segmented dimension indicator layer includes at least one of a product dimension, a customer group dimension, and a time dimension, and each dimension supports hierarchical nested drilling.

3. The indicator data analysis method according to claim 2, characterized in that: The step of obtaining the business scenario characteristics of the indicator data and classifying the indicator data by scenarios in combination with preset indicator coding rules includes: Acquire business scenario features of the indicator data, and identify scenario tags of the indicator data according to the business scenario features; Based on a preset indicator coding rule, a unique identifier is generated by combining the scene label and the level label; wherein the unique identifier includes at least one of a level code and a scene classification code; Based on the unique identifier, the indicator data having the same scene classification code is stored in an independent physical table of a distributed column-based database.

4. The indicator data analysis method according to claim 3, wherein: The physically isolating the storage results of the indicator data in the distributed column database physical table of the scenario classification based on the hierarchical label includes: in the distributed column database physical table, based on the partition key mechanism of the distributed column database, allocating the indicator data with the same hierarchical code in the unique identifier to the same partition.

5. The indicator data analysis method according to claim 4, characterized in that: The steps of obtaining business scenario features of the indicator data, classifying the indicator data by scenario in combination with preset indicator coding rules, and physically isolating the storage results of the indicator data in the physical table of the distributed column-based database of the scenario classification based on the hierarchical label also include: When a new business scenario is added, the scenario tag of the indicator data of the new business scenario is identified, and the level tag corresponding to the indicator data of the new business scenario is associated; Based on the scenario label and the associated level label of the indicator data, a unique identifier is generated according to a preset indicator coding rule; wherein the scenario classification code in the unique identifier is a newly added scenario classification code; An independent distributed column-based database physical table is created according to the newly added scenario classification code, and a storage area of ​​the independent distributed column-based database physical table is divided using the partition key mechanism.

6. The indicator data analysis method according to claim 5, characterized in that: The step of selecting target indicator data and data display parameters based on the user interface, and querying the target indicator data in the physically isolated storage results to generate a real-time indicator dashboard includes: Select target indicator data through the online EXCEL interface to trigger the preset indicator coding rule analysis; Based on the analysis results of the target indicator data, the corresponding partitions in the physical table of the distributed column-based database are associated and matched, and the target indicator data is queried in real time; Set data display parameters through the online EXCEL interface and build EXCEL templates; After the EXCEL template is built, the target indicator data is filled in in real time and rendered to generate a real-time indicator dashboard.

7. The indicator data analysis method according to claim 6, characterized in that: The step of querying the target indicator data in real time includes: Obtaining the hierarchical code and scene classification code from the parsed results of the target indicator data; Triggering the pre-aggregation mechanism of the distributed storage engine to locate the target indicator data in the physical table and corresponding partition of the distributed column database through the hierarchical coding and scenario classification code; Extract compressed target indicator data based on positioning information.

8. The indicator data analysis method according to claim 7, wherein: After the step of selecting target indicator data and data display parameters based on the user interface and querying the target indicator data in the physically isolated storage result to generate a real-time indicator dashboard, the following steps are included: According to the drill-down link sequence configured by the user, the scenario classification code and hierarchical code of the target indicator data in the drill-down link are parsed; Associating the target indicator data in the drill-down link with the physical table through a preset code-physical table mapping rule; If the drill-down link of the target sequence requires association with physical tables of different scenario classification codes, it is considered a cross-scenario association, triggering an asynchronous extract-transform-load (ETL) link to complete the missing target indicator data; Otherwise, the target indicator data is queried in real time to generate a drill-down link in the target order.

9. An indicator data analysis device, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the indicator data analysis method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the indicator data analysis method according to any one of claims 1 to 7 are implemented.

Images