A system and method for multi-dimensional data correlation analysis based on unified modeling

Abstract

The application discloses a system and method for multi-dimensional data correlation analysis based on unified modeling, and relates to the technical field of intelligent analysis of big data; the system comprises the following steps: step 1, semantic model construction is performed; step 2, query compilation and optimization are performed; step 3, query execution and scheduling are performed; step 4, interactive analysis and visualization are performed; end-to-end full-process visual display is provided by using a multi-dimensional report designer MRD, and interactive drilling and narrative analysis are supported; and step 5, closed-loop optimization is performed; the complexity of a bottom-layer data structure is shielded by the semantic model, automatic correlation analysis across fact tables is supported, and thus a business personnel can independently complete deep data exploration and decision support without mastering professional techniques such as SQL.

Description

TECHNICAL FIELD

[0001] The application discloses a system and method for multi-dimensional data correlation analysis based on unified modeling, and relates to the technical field of intelligent analysis of big data. BACKGROUND

[0002] In enterprise data analysis, data is usually stored in multiple fact tables such as a sales fact table, a stock fact table, and a user behavior fact table, and these fact tables are correlated with each other through shared dimension tables such as a time dimension, a product dimension, and a user dimension table. Existing analysis methods mainly rely on a star / snowflake model of a data warehouse and SQL queries. However, this approach still has limitations, such as:

[0003] 1. The model is rigid and slow to respond to business changes. When a new analysis dimension or indicator is proposed by the business, a data engineer needs to modify the data model and the ETL process, and the process is long and cannot respond quickly.

[0004] 2. Multi-fact table correlation analysis is complex. If a business person needs to analyze an indicator across fact tables, such as "analyze the sales conversion rate of high inventory goods", a complex SQL statement containing multiple JOINs needs to be written, which has a high technical threshold and is prone to errors.

[0005] 3. The report development cycle is long: even if there is a BI tool, it still takes technical personnel to develop a new multi-dimensional analysis report, especially a drill-down report, and business personnel cannot independently conduct exploratory analysis, which seriously hinders the efficiency of data-driven decision-making.

[0006] Although there are some self-service BI tools on the market, they only provide drag-and-drop functions, and their underlying structure usually relies on a single wide table or a pre-defined data model, which still lacks flexible support for dynamic and multi-fact table correlation scenarios, and cannot achieve true cross-model and cross-fact table free drilling and correlation analysis. SUMMARY

[0007] To solve the problems of the prior art, the application provides a system and method for multi-dimensional data correlation analysis based on unified modeling, which shields the complexity of the underlying data structure through a semantic model, supports automatic correlation analysis across fact tables, and enables business personnel to independently complete deep data exploration and decision support without mastering professional techniques such as SQL.

[0008] The specific scheme provided by the application is as follows:

[0009] The application provides a method for multi-dimensional data correlation analysis based on unified modeling, comprising:

[0010] Step 1: Construct a semantic model

[0011] Connect multi-source data: connect multi-source heterogeneous databases through interfaces, automatically scan data table structures,

[0012] Intelligent recommendation of association relationship: according to the scanning results, the main foreign key recommendation, field name similarity and data sampling analysis are carried out, the association relationship between dimensions and facts is obtained, and the association relationship is visualized in the form of a graph,

[0013] Define dimension level and index: define dimension level, set index aggregation rule and calculation expression,

[0014] According to the association relationship between the dimensions and the facts, the dimension level index aggregation rule and the calculation expression form a semantic model;

[0015] Step 2: query compilation and optimization:

[0016] Parse query intent: parse user query intent, get query intent,

[0017] Multi-objective cost optimization: according to the query intent, use the semantic model to find the association path between multiple fact tables and dimension tables, generate multiple candidate physical execution plans according to the association path, and select the optimized physical execution plan output,

[0018] Step 3: query execution and scheduling:

[0019] Multi-engine adaptation: build a multi-engine adaptation layer, select an execution engine according to the physical execution plan,

[0020] Step 4: interactive analysis and visualization: use multi-dimensional report designer MRD to provide end-to-end full-process visual display, and support interactive drilling and narrative analysis,

[0021] Step 5: closed-loop optimization:

[0022] Monitoring and feedback: collect query performance, resource usage and user behavior data,

[0023] Automatic optimization model and plan: according to the collected data, the semantic model and the physical execution plan are automatically optimized regularly.

[0024] Further, the step 1 of the method for multi-dimensional data association analysis based on unified modeling comprises:

[0025] Version unification and consistency check: the semantic model is stored in a versioned manner, all semantic model changes are recorded with version number, change content and personnel information, rollback and difference comparison are supported, semantic consistency check of the semantic model is performed, index ambiguity, circular dependency and logic conflict are automatically detected, and the semantic model is ensured to be rigorous and reliable.

[0026] Further, the step 2 of the method for multi-dimensional data correlation analysis based on unified modeling performs distributed cost evaluation on the candidate physical execution plans, including:

[0027] TotalCost=a*(SF / DV)+b*CL+c*(1-CH);

[0028] TotalCost=a*(SF / DV)+b*CL+c*(1-CH);

[0029] DV is an abbreviation of DataVolume, indicating the amount of data to be processed, reflecting data reading or transmission overhead,

[0030] SF is an abbreviation of ScaleFactor, indicating a preset data scaling factor, which is adjusted based on cluster benchmark performance and used for normalizing data volume influence,

[0031] CL is an abbreviation of ClusterLoad, indicating a cluster load rate ranging from 0 to 1, reflecting the current cluster resource competition degree,

[0032] CH is an abbreviation of CacheHitRate, indicating a cache hit rate ranging from 0 to 1,

[0033] a, b, and c are weight coefficients, which are adjusted through actual scenes and satisfy a+b+c=1;

[0034] Some candidate physical execution plans with smaller total cost are selected, nodes in a distributed environment are coordinated, local costs of each node are collected, and the local costs of each node are aggregated into a global cost,

[0035] The physical execution plan with the smallest global cost is selected as the output physical execution plan after optimization.

[0036] Further, the step 2 of the method for multi-dimensional data correlation analysis based on unified modeling also generates materialized views and caches: for high-frequency queries, incremental materialized views are generated, and incremental updates are triggered based on logs or timestamps, ensuring the balance between query performance and data real-time performance,

[0037] Query rewriting and physical execution plan caching are also performed, physical execution plans are automatically reused for semantically equivalent queries, and repeated optimization overhead is reduced.

[0038] Further, the step 3 of the method is to perform the cross-fact table association query in the physical execution plan in stages to balance the row load: first, perform the filtering and aggregation operations on each fact table in parallel, and then perform the join operation on the intermediate results,

[0039] Simultaneous failover and monitoring: real-time monitoring of query execution status, failover using dynamic resource allocation, automatic response to node failure or load skew, ensuring high availability and stability of queries.

[0040] The application also provides a system for multi-dimensional data association analysis based on unified modeling, comprising a semantic model construction module, a compilation module, an execution module, a visualization management module, and a feedback module,

[0041] The semantic model construction module performs semantic model construction:

[0042] Connecting multiple sources of data: connecting multiple heterogeneous databases through interfaces, automatically scanning data table structures,

[0043] Intelligent recommendation of association relationships: based on the scanning results, performing primary key recommendation, field name similarity, and data sampling analysis to obtain the association relationships between dimensions and facts, and visualizing the association relationships in the form of a graph,

[0044] Defining dimension levels and indicators: defining dimension levels, setting indicator aggregation rules and calculation expressions,

[0045] Forming a semantic model based on the association relationships between dimensions and facts, dimension level indicator aggregation rules, and calculation expressions;

[0046] The compilation module performs query compilation and optimization:

[0047] Analyzing query intent: analyzing user query intent to obtain query intent,

[0048] Multi-target cost optimization: based on the query intent, using the semantic model to find association paths between multiple fact tables and dimension tables, generating multiple candidate physical execution plans based on the association paths, performing distributed cost evaluation optimization on the candidate physical execution plans, and selecting the optimized physical execution plan for output,

[0049] The execution module performs query execution and scheduling:

[0050] Multi-engine adaptation: building a multi-engine adaptation layer to select an execution engine based on the physical execution plan,

[0051] The visualization management module performs interactive analysis and visualization: using the multi-dimensional report designer MRD to provide end-to-end full-process visual display, and supporting interactive drilling and narrative analysis,

[0052] The feedback module performs closed-loop optimization:

[0053] Monitoring and feedback: Collect query performance, resource usage and user behavior data,

[0054] Automatic optimization model and plan: Regularly automatically optimize semantic model and physical execution plan according to collected data.

[0055] Further, the semantic model construction module of the system for multi-dimensional data correlation analysis based on unified modeling manages the semantic model, including:

[0056] Version unification and consistency check: Store the semantic model in a versioned storage manner, record version number, change content and personnel information for all semantic model changes, support rollback and difference comparison, and perform semantic consistency check of the semantic model, automatically detect index ambiguity, circular dependency and logic conflict, and ensure that the semantic model is rigorous and reliable.

[0057] Further, the compiling module of the system for multi-dimensional data correlation analysis based on unified modeling performs distributed cost evaluation on the candidate physical execution plan, including:

[0058] Calculate the total cost TotalCost for each candidate physical execution plan, the formula is:

[0059] TotalCost=a*(SF / DV)+b*CL+c*(1-CH);

[0060] Where DV is DataVolume, which represents the amount of data to be processed, reflecting data reading or transmission overhead,

[0061] SF is ScaleFactor, which represents a preset data scaling factor, adjusted based on cluster benchmark performance, used for normalizing data volume impact,

[0062] CL is ClusterLoad, which represents the cluster load rate, ranging from 0 to 1, reflecting the current cluster resource competition degree,

[0063] CH is CacheHitRate, which represents the cache hit rate, ranging from 0 to 1,

[0064] a, b, c: weight coefficients, adjusted through actual scenarios, satisfying a+b+c=1;

[0065] Select some candidate physical execution plans with smaller total cost, in a distributed environment, coordinate nodes, collect local costs of each node, and aggregate local costs of each node into global cost,

[0066] The physical execution plan with the minimum global cost is selected as the physical execution plan of the optimized output.

[0067] Further, the compiling module of the system for multi-dimensional data correlation analysis based on unified modeling further generates a materialized view and caches: for high-frequency queries, an incremental materialized view is generated, and incremental updating is triggered based on logs or timestamps, ensuring the balance between query performance and data real-time performance.

[0068] Query rewriting and physical execution plan caching are also performed, and the physical execution plan is automatically reused for semantically equivalent queries, reducing repeated optimization overhead.

[0069] Further, the executing module of the system for multi-dimensional data correlation analysis based on unified modeling performs staged execution for correlation queries across fact tables in the physical execution plan to balance row loads: filtering and aggregation operations are first performed in parallel on each fact table, and then join operations are performed on the intermediate results.

[0070] Fault transfer and monitoring are also performed: the query execution state is monitored in real time, fault transfer is performed using dynamic resource allocation, and automatic responses to node failures or load tilting are ensured to ensure query high availability and stability.

[0071] The benefits of the present application are:

[0072] A business-friendly unified semantic model is constructed, complex physical data models are abstracted into business terms such as indicators and dimensions, and the association relationship is clearly defined, reducing the understanding cost of business personnel;

[0073] Automatic intelligent association of multiple fact tables is realized, and according to the association relationship defined in the semantic model, the join, aggregation and drill-down analysis between multiple fact tables are automatically processed without manual writing of complex SQL;

[0074] Zero-code multi-dimensional report generation capability is provided, and business users can independently and flexibly configure dimensions and indicators through completely visual drag-and-drop operations to generate and interactively explore multi-dimensional analysis reports in real time;

[0075] The query performance of massive data is ensured, and through model compilation optimization, pre-computation, distributed query and other technologies, complex queries based on the semantic layer can be quickly responded to;

[0076] In summary, the present application realizes agile delivery from data model to analysis application, hides the complexity of the underlying data structure through the semantic model, supports automatic correlation analysis across fact tables, and enables business personnel to independently complete deep data exploration and decision support without mastering professional techniques such as SQL. BRIEF DESCRIPTION OF DRAWINGS

[0077] Figure 1is a schematic diagram of the method flow of the present application. DETAILED DESCRIPTION

[0078] The present application will be further described below in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the present application and implement it. The embodiments are not intended to limit the present application.

[0079] Embodiment 1: The present application provides a method for multi-dimensional data correlation analysis based on unified modeling, comprising:

[0080] Step 1: Semantic model construction and management:

[0081] Connecting multi-source data: connecting multi-source heterogeneous databases through JDBC / ODBC or data lake interface, automatically scanning data table structure,

[0082] Intelligent recommendation of correlation relationship: according to the scanning results, performing primary key recommendation, field name similarity and data sampling analysis, obtaining the correlation relationship between dimensions and facts, and visualizing the correlation relationship in the form of a graph,

[0083] Defining dimension hierarchy and indicators: defining dimension hierarchy such as time: year-> season-> month-> day, setting indicator aggregation rules such as SUM / AVG, etc., and calculating expressions such as "gross profit rate = (revenue-cost) / revenue", etc.,

[0084] Forming a semantic model according to the correlation relationship between dimensions and facts, dimension hierarchy, indicator aggregation rules and calculation expressions,

[0085] Version unification and consistency check: storing the semantic model in a versioned storage manner, recording version number, change content and personnel information for all semantic model changes, supporting rollback and difference comparison, and performing semantic consistency check of the semantic model, automatically detecting indicator ambiguity, circular dependency and logical conflict, and ensuring the semantic model to be rigorous and reliable.

[0086] Step 2: Query compilation and optimization:

[0087] Parsing query intent: parsing user query intent, obtaining query intent,

[0088] Multi-target cost optimization: according to the query intent, using the semantic model to find correlation paths between multiple fact tables and dimension tables, generating multiple candidate physical execution plans according to the correlation paths, performing distributed cost evaluation optimization on the candidate physical execution plans, and selecting the optimized physical execution plan for output.

[0089] Among them, the distributed cost evaluation of the candidate physical execution plan includes:

[0090] TotalCost = a * (SF / DV) + b * CL + c * (1-CH);

[0091] TotalCost = a * (SF / DV) + b * CL + c * (1-CH);

[0092] where DV is the abbreviation of DataVolume, representing the amount of data to be processed, reflecting the data read or transmission overhead,

[0093] SF is the abbreviation of ScaleFactor, representing the preset data scaling factor, which is adjusted based on the cluster benchmark performance, and is used to normalize the data volume impact,

[0094] CL is the abbreviation of ClusterLoad, representing the cluster load rate, ranging from 0 to 1, reflecting the current cluster resource competition degree,

[0095] CH is the abbreviation of CacheHitRate, representing the cache hit rate, ranging from 0 to 1,

[0096] a, b, c: weight coefficients, adjusted through actual scenarios, satisfying a+b+c=1;

[0097] Select some candidate physical execution plans with smaller total cost, in a distributed environment, coordinate nodes, collect local costs of each node, and aggregate local costs of each node into global cost,

[0098] Select the physical execution plan with the smallest global cost as the output physical execution plan after optimization.

[0099] Also generate materialized views and cache: for high-frequency queries, generate incremental materialized views and trigger incremental updates based on logs or timestamps to ensure the balance between query performance and data real-time performance,

[0100] Also perform query rewriting and physical execution plan caching, automatically reuse physical execution plans for semantically equivalent queries, and reduce repeated optimization overhead.

[0101] Step 3: Query execution and scheduling:

[0102] Multi-engine adaptation: build a multi-engine adaptation layer such as Presto, Spark and Druid query engine, select the execution engine execution plan according to the complexity of the query SQL involved in the physical execution plan, data size and SLA timeliness requirements, such as Presto for interactive queries, Spark for ETL processing, Druid for pre-aggregation queries, etc.

[0103] For the association query across the fact table in the physical execution plan, the phased execution is performed to balance the row load: the filtering and aggregation operations are performed on each fact table in parallel, and then the join operation is performed on the intermediate results, so that the Shuffle data amount can be greatly reduced.

[0104] Simultaneous failover and monitoring: real-time monitoring of query execution status, failover using dynamic resource allocation, automatic response to node failure or load skew, ensuring query high availability and stability.

[0105] Step 4: Interactive analysis and visualization, end-to-end full-process visualization provided by multi-dimensional report designer MRD, and support for interactive drilling and narrative analysis:

[0106] Drag to generate query: dimensions and indicators can be dragged to the canvas to generate query statements in real time using multi-dimensional report designer MRD, and automatically recommend visualization schemes such as time series data recommending line chart and proportion data recommending pie chart,

[0107] Natural language parsing: users can directly generate reports through natural language input, and parse query intent based on NLP technology and map to semantic model,

[0108] Intelligent visualization and drilling: when performing interactive drilling and narrative analysis, clicking on any area of the chart such as a monthly column chart automatically drills down to daily granularity data, records the drilling path to generate an analysis story line, supports multi-user collaborative editing, commenting and sharing, and can be exported to PDF or published to a large screen with one key.

[0109] Step 5: Closed-loop optimization:

[0110] Monitoring and feedback: collect query performance, resource usage and user behavior data,

[0111] Automatic optimization model and plan: according to the collected data, periodically automatically optimize the semantic model and physical execution plan, such as adjusting the materialized view strategy, updating the statistical information, and optimizing the association path. At the same time, the model health score and optimization suggestions can be provided to help administrators continuously improve the model quality and analysis experience, forming a closed-loop self-optimization.

[0112] Embodiment 2: The application also provides a multi-dimensional data association analysis system based on unified modeling, comprising a semantic model construction module, a compilation module, an execution module, a visualization management module and a feedback module,

[0113] The semantic model construction module constructs a semantic model:

[0114] Connect multiple sources of data: connect multiple heterogeneous databases through interfaces, automatically scan data table structures,

[0115] Intelligent recommendation association: according to the scanning result, the primary-foreign key recommendation, the field name similarity and the data sampling analysis are carried out, the association between the dimensions and the facts is obtained, and the association is visualized in the form of a graph,

[0116] Definition of dimension level and index: the dimension level is defined, the index aggregation rule and the calculation expression are set,

[0117] According to the association between the dimensions and the facts, the dimension level index aggregation rule and the calculation expression, a semantic model is formed;

[0118] The query compilation and optimization are carried out by the compiling module:

[0119] Analysis of query intention: the query intention is analyzed, the query intention is obtained,

[0120] Multi-target cost optimization: according to the query intention, the semantic model is used to find the association path between multiple fact tables and dimension tables, multiple candidate physical execution plans are generated according to the association path, the distributed cost evaluation optimization is carried out on the candidate physical execution plans, and the optimized physical execution plan is selected and output,

[0121] The query execution and scheduling are carried out by the execution module:

[0122] Multi-engine adaptation: a multi-engine adaptation layer is constructed, and the execution engine execution plan is selected according to the physical execution plan,

[0123] The interactive analysis and visualization are carried out by the visual management module: the end-to-end full-process visual display is provided by the multi-dimensional report designer MRD, and the interactive drilling and narrative analysis are supported,

[0124] The feedback module carries out closed-loop optimization:

[0125] Monitoring and feedback: the query performance, resource usage and user behavior data are collected,

[0126] Automatic optimization model and plan: according to the collected data, the semantic model and the physical execution plan are automatically optimized regularly.

[0127] The information interaction and execution process between the modules in the above system are based on the same concept as the method embodiments of the present application, and the specific content can be referred to the description in the method embodiments of the present application, which will not be repeated here.

[0128] Similarly, the benefits of the system of the present application are:

[0129] A business-friendly unified semantic model is constructed, a complex physical data model is abstracted into business terms such as indexes and dimensions, and the association relationship is defined, thereby reducing the understanding cost of business personnel;

[0130] The automatic intelligent association of multi-fact tables is realized, and according to the association relationship defined in the semantic model, the connection, aggregation and down-drilling analysis among the multi-fact tables are automatically processed, without manually writing complex SQL;

[0131] A zero-code multidimensional report generation capability is provided, and through completely visualized drag-and-drop operation, a business user can independently and flexibly configure dimensions and indexes, and instantly generates and interactively explores multidimensional analysis reports;

[0132] The query performance of massive data is ensured, through model compilation optimization, pre-computation, distributed query and other technologies, and it is ensured that complex queries based on the semantic layer can be quickly responded;

[0133] In summary, the system of the present application realizes agile delivery from a data model to an analysis application, the complexity of a bottom-layer data structure is shielded through a semantic model, automatic association analysis across fact tables is supported, and a business person can independently complete deep data exploration and decision support without mastering professional technologies such as SQL.

[0134] It should be noted that not all steps and modules in the above processes and system structures are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed, and can be adjusted according to needs. The system structure described in the above embodiments can be a physical structure or a logical structure, that is, some modules can be implemented by the same physical entity, or some modules can be implemented by multiple physical entities, or can be implemented by some components in multiple independent devices.

[0135] The above-described embodiments are only preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Equivalent substitutions or transformations made by those skilled in the art on the basis of the present application are within the protection scope of the present application. The protection scope of the present application is subject to the claims.

Claims

1. A method for multidimensional data correlation analysis based on unified modeling, characterized by: include: Step 1: Construct a semantic model: Connecting to multi-source data: Connecting to heterogeneous databases from multiple sources via an interface, automatically scanning the data table structure. Intelligent recommendation of associations: Based on the scanning results, primary and foreign key recommendations, field name similarity analysis, and data sampling analysis are performed to obtain the associations between dimensions and facts, and the associations are visualized in the form of a graph. Define dimensional hierarchy and metrics: Define the dimensional hierarchy, set the metric aggregation rules and calculation expressions. A semantic model is formed based on the relationship between dimensions and facts, the aggregation rules of dimension-level indicators, and the calculation expressions. Step 2: Perform query compilation and optimization: Parsing query intent: Parsing the user's query intent to obtain the query intent. Multi-objective cost optimization is performed: based on the query intent, a semantic model is used to find association paths between multiple fact tables and dimension tables. Multiple candidate physical execution plans are generated based on the association paths. Distributed cost evaluation and optimization are performed on the candidate physical execution plans, and the optimized physical execution plan is output. Step 3: Perform query execution and scheduling: Multi-engine adaptation: Construct a multi-engine adaptation layer to select the execution engine execution plan based on the physical execution plan. Step 4: Perform interactive analysis and visualization: Utilize the Multidimensional Report Designer (MRD) to provide end-to-end visualization of the entire process, supporting interactive drill-down and narrative analysis. Step 5: Perform closed-loop optimization: Monitoring and feedback: Collect data on query performance, resource usage, and user behavior. Automatic model and plan optimization: The semantic model and physical execution plan are automatically optimized periodically based on the collected data.

2. The method for multidimensional data correlation analysis based on unified modeling according to claim 1, characterized in that: Step 1 involves managing the semantic model, including: Perform version unification and consistency checks: The semantic model is stored in a versioned storage manner. All changes to the semantic model are recorded with version number, change content and personnel information. Rollback and difference comparison are supported. At the same time, semantic consistency checks are performed on the semantic model, automatically detecting ambiguity of indicators, circular dependencies and logical conflicts to ensure that the semantic model is rigorous and reliable.

3. The method for multidimensional data correlation analysis based on unified modeling according to claim 1, characterized in that: Step 2 involves a distributed cost evaluation of the candidate physical execution plans, including: Calculate the total cost (TotalCost) for each candidate physical execution plan using the following formula: TotalCost=a*(SF / DV)+b*CL+c*(1-CH); DV stands for DataVolume, representing the amount of data to be processed and reflecting the overhead of data reading or transmission. SF is an abbreviation for ScaleFactor, representing a preset data scaling factor. It is adjusted based on the cluster's baseline performance and is used to normalize the impact of data volume. CL is an abbreviation for ClusterLoad, representing the cluster load rate, ranging from 0 to 1, and reflecting the current level of resource contention within the cluster. CH is an abbreviation for CacheHitRate, representing the cache hit rate, ranging from 0 to 1. a, b, c: weighting coefficients, adjusted according to the actual scenario to satisfy a+b+c=1; Select a few candidate physical execution plans with smaller total costs. In a distributed environment, coordinate nodes, collect the local costs of each node, and aggregate the local costs of each node into a global cost. The physical execution plan with the lowest global cost is selected as the optimized physical execution plan.

4. The method for multidimensional data correlation analysis based on unified modeling according to claim 1, characterized in that: Step 2 also generates and caches materialized views: For high-frequency queries, incremental materialized views are generated, and incremental updates are triggered based on logs or timestamps to ensure a balance between query performance and data real-time performance. It also performs query rewriting and physical execution plan caching, automatically reusing physical execution plans for semantically equivalent queries to reduce redundant optimization overhead.

5. The method for multidimensional data correlation analysis based on unified modeling according to claim 1, characterized in that: In step 3, for cross-fact table join queries in the physical execution plan, the execution is carried out in stages to achieve load balancing: first, filtering and aggregation operations are performed in parallel on each fact table, and then join operations are performed on the intermediate results. Simultaneously, failover and monitoring are performed: query execution status is monitored in real time, failover is performed using dynamic resource allocation, and node failures or load imbalances are automatically addressed to ensure high availability and stability of queries.

6. A system for multidimensional data correlation analysis based on unified modeling, characterized by: It includes a semantic model building module, a compilation module, an execution module, a visualization management module, and a feedback module. The semantic model building module constructs the semantic model: Connecting to multi-source data: Connecting to heterogeneous databases from multiple sources via an interface, automatically scanning the data table structure. Intelligent recommendation of associations: Based on the scanning results, primary and foreign key recommendations, field name similarity analysis, and data sampling analysis are performed to obtain the associations between dimensions and facts, and the associations are visualized in the form of a graph. Define dimensional hierarchy and metrics: Define the dimensional hierarchy, set the metric aggregation rules and calculation expressions. A semantic model is formed based on the relationship between dimensions and facts, the aggregation rules of dimension-level indicators, and the calculation expressions. The compilation module performs query compilation and optimization: Parsing query intent: Parsing the user's query intent to obtain the query intent. Multi-objective cost optimization is performed: based on the query intent, a semantic model is used to find association paths between multiple fact tables and dimension tables. Multiple candidate physical execution plans are generated based on the association paths. Distributed cost evaluation and optimization are performed on the candidate physical execution plans, and the optimized physical execution plan is output. The execution module performs query execution and scheduling: Multi-engine adaptation: Construct a multi-engine adaptation layer to select the execution engine execution plan based on the physical execution plan. The visualization management module enables interactive analysis and visualization: Utilizing the multidimensional report designer (MRD), it provides end-to-end, full-process visualization and supports interactive drill-down and narrative analysis. The feedback module undergoes closed-loop optimization: Monitoring and feedback: Collect data on query performance, resource usage, and user behavior. Automatic model and plan optimization: The semantic model and physical execution plan are automatically optimized periodically based on the collected data.

7. A system for multidimensional data correlation analysis based on unified modeling according to claim 6, characterized in that: The semantic model building module manages the semantic model, including: Perform version unification and consistency checks: The semantic model is stored in a versioned storage manner. All changes to the semantic model are recorded with version number, change content and personnel information. Rollback and difference comparison are supported. At the same time, semantic consistency checks are performed on the semantic model, automatically detecting ambiguity of indicators, circular dependencies and logical conflicts to ensure that the semantic model is rigorous and reliable.

8. A system for multidimensional data correlation analysis based on unified modeling according to claim 6, characterized in that: The compilation module performs a distributed cost evaluation of the candidate physical execution plans, including: Calculate the total cost (TotalCost) for each candidate physical execution plan using the following formula: TotalCost=a*(SF / DV)+b*CL+c*(1-CH); DV stands for DataVolume, representing the amount of data to be processed and reflecting the overhead of data reading or transmission. SF is an abbreviation for ScaleFactor, representing a preset data scaling factor. It is adjusted based on the cluster's baseline performance and is used to normalize the impact of data volume. CL is an abbreviation for ClusterLoad, representing the cluster load rate, ranging from 0 to 1, and reflecting the current level of resource contention within the cluster. CH is an abbreviation for CacheHitRate, representing the cache hit rate, ranging from 0 to 1. a, b, c: weighting coefficients, adjusted according to the actual scenario to satisfy a+b+c=1; Select a few candidate physical execution plans with smaller total costs. In a distributed environment, coordinate nodes, collect the local costs of each node, and aggregate the local costs of each node into a global cost. The physical execution plan with the lowest global cost is selected as the optimized physical execution plan.

9. A system for multidimensional data correlation analysis based on unified modeling according to claim 6, characterized in that: The compilation module also generates and caches materialized views: for high-frequency queries, it generates incremental materialized views and triggers incremental updates based on logs or timestamps, ensuring a balance between query performance and data real-time performance. It also performs query rewriting and physical execution plan caching, automatically reusing physical execution plans for semantically equivalent queries to reduce redundant optimization overhead.

10. A system for multidimensional data correlation analysis based on unified modeling according to claim 6, characterized in that: The execution module performs cross-fact table join queries in the physical execution plan in stages to achieve load balancing: first, filtering and aggregation operations are performed in parallel on each fact table, and then join operations are performed on the intermediate results. Simultaneously, failover and monitoring are performed: query execution status is monitored in real time, failover is performed using dynamic resource allocation, and node failures or load imbalances are automatically addressed to ensure high availability and stability of queries.

Images