A data query method and related products

Abstract

The application discloses a data query method and related products, the method comprises the following steps: obtaining index metadata through an index mode; the index mode comprises a visual configuration index mode or a formula definition index mode; based on the association table information in the index metadata, an associated wide table is constructed; based on the associated wide table, the upstream table data is updated synchronously to obtain associated data; based on the index metadata and the associated data, a materialized view is constructed; the materialized view is constructed based on the index metadata, the associated data and a pre-computed result; the pre-computed result is obtained by pre-computing based on the index metadata and the associated data; a query request is matched with the materialized view, and a pre-computed result matched with the query request is determined as a final result. The application realizes an end-to-end closed loop from index definition, data preparation to accelerated query, effectively solves the problems of poor flexibility, low performance, high maintenance cost and the like in a traditional scheme.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data query method and related products. Background Technology

[0002] Current data query systems generally suffer from problems such as limited modes, insufficient flexibility, and performance bottlenecks in indicator acquisition. Mainstream solutions typically adopt a polarized design: one type is visual drag-and-drop BI tools, which are easy to operate but only support preset aggregation logic and are difficult to express complex business formulas (such as multi-level nested calculations, conditional weighting, cross-period comparisons, etc.); the other type is data warehouse tools based on SQL or scripts, which have powerful expressive capabilities but require a high technical threshold for users, making them difficult for business personnel to use independently, resulting in low analysis efficiency.

[0003] In multi-table join scenarios, systems generally rely on real-time JOIN operations to integrate metrics. When large tables or multiple high-cardinality dimension tables are involved, query latency increases dramatically, severely impacting the user experience. To alleviate this problem, some platforms adopt a wide-table pre-join model, but its structure is rigid. Once the business needs to add dimensions or adjust the granularity, it requires remodeling and ETL, resulting in high iteration costs, slow response times, and difficulty in adapting to rapidly changing analytical needs. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a data query method and related products, aiming to improve the ease of use, expressiveness, and flexibility of data queries.

[0005] The embodiments of this application disclose the following technical solutions: The first aspect of this application provides a data query method, including: Indicator metadata is obtained through indicator patterns; the indicator patterns include visually configurable indicator patterns or formulaically defined indicator patterns. Construct a wide relational table based on the relational table information in the indicator metadata; Based on the synchronization of upstream table data updates using the aforementioned associated wide table, associated data is obtained; A materialized view is constructed based on the indicator metadata and the associated data; the materialized view is constructed based on the indicator metadata, the associated data, and the pre-calculated results; the pre-calculated results are obtained by pre-calculating based on the indicator metadata and the associated data. The query request is matched with the materialized view, and the pre-calculated result that matches the query request is determined as the final result.

[0006] Optionally, obtaining indicator metadata through indicator patterns specifically includes: If the indicator mode is the visual configuration indicator mode, then develop a drag-and-drop interface and determine the drag operation; Quickly create metrics and obtain metric data using drag-and-drop operations; If the indicator pattern is the formula-defined indicator pattern, then develop a syntax parser to determine the formula logic writing data; Based on the aforementioned formula logic, data is written to quickly create indicators and obtain indicator data; The indicator data is standardized to obtain indicator metadata.

[0007] Optionally, the standardization process of the indicator data to obtain indicator metadata specifically includes: If the indicator data is determined through the visualization configuration indicator mode, then the indicator data is converted into standardized calculation logic to obtain indicator metadata; If the indicator data is determined through the formulaic definition of the indicator pattern, then the indicator data is parsed into execution logic to obtain indicator metadata.

[0008] Optionally, the method further includes: Rules for obtaining permissions; The permission rules are embedded into the materialized view.

[0009] Optionally, the step of matching the query request with the materialized view and determining the pre-calculated result matching the query request as the final result specifically includes: Determine query permissions based on the query request; The query permissions are verified to obtain the verification result and permission identifier; If the verification result passes, materialized view matching is performed based on the permission identifier, and the materialized view that matches the permission identifier is taken as the target materialized view. The query request is parsed to extract the query logic. If the query filtering conditions in the query logic are a subset of the filtering conditions in the target materialized view and the query granularity in the query logic satisfies the roll-up generation conditions of the target materialized view, then the pre-calculated result of the corresponding target materialized view is extracted as the final result.

[0010] Optionally, the method further includes: If no pre-calculated result is found to match the query request, the corresponding associated wide table and indicator metadata are called based on the query request to perform real-time calculation and obtain the final result.

[0011] Optionally, the method further includes: Real-time acquisition of updated and adjusted data; the updated and adjusted data includes at least one of the following: permission rule update data, indicator metadata update data, and related wide table structure update data; The materialized view is updated in real time based on the updated adjustment data.

[0012] Optionally, the method further includes: Periodically obtain usage data for the materialized view; the usage data includes the number of hits. The materialized view is cleaned based on the usage data.

[0013] A second aspect of this application provides a data query apparatus, comprising: The dual-mode indicator definition module is used to obtain indicator metadata through indicator modes; the indicator modes include a visually configurable indicator mode or a formulaically defined indicator mode. The table association management module is used to construct a wide association table based on the association table information in the indicator metadata; and to synchronize the upstream table data update based on the wide association table to obtain the association data. The intelligent pre-calculation module is used to construct a materialized view based on the indicator metadata and the associated data; the materialized view is constructed based on the indicator metadata, associated data, and pre-calculation results; the pre-calculation results are obtained by pre-calculating based on the indicator metadata and the associated data. The query execution module is used to match the query request with the materialized view and determine the pre-calculated result that matches the query request as the final result.

[0014] A third aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the data query method provided in the first aspect.

[0015] Compared with the prior art, this application has the following beneficial effects: This application includes obtaining indicator metadata through indicator patterns; the indicator patterns include visual configuration indicator patterns or formulaic definition indicator patterns; constructing a related wide table based on the related table information in the indicator metadata; synchronizing upstream table data updates based on the related wide table to obtain related data; constructing a materialized view based on the indicator metadata and the related data; the materialized view is constructed based on the indicator metadata, the related data, and pre-calculated results; the pre-calculated results are obtained by pre-calculating based on the indicator metadata and the related data; matching a query request with the materialized view, and determining the pre-calculated result that matches the query request as the final result.

[0016] This application generates consistent indicator metadata through a unified indicator model (including a visually configurable indicator model and a formulaic indicator definition model), ensuring logical alignment and consistent results for different user groups when defining indicators, balancing business usability and technical flexibility. Based on the explicit association table information in the indicator metadata, targeted association wide tables can be built in advance, avoiding the need for high-overhead multi-table JOIN operations during queries. Simultaneously, by monitoring data changes in upstream tables, synchronous updates of association wide tables are achieved, ensuring the timeliness and consistency of association data. Building upon this, materialized views are automatically constructed by combining indicator metadata, real-time association data, and pre-calculated results. These materialized views encapsulate aggregation structures organized according to business semantics, supporting efficient queries. When a user initiates a query request, it is intelligently matched with registered materialized views. If a match is successful, the corresponding pre-calculated result is returned directly, eliminating the need for recalculation. This application significantly reduces query latency and computational resource consumption, improves system concurrency, and achieves an end-to-end closed loop from indicator definition and data preparation to accelerated queries through a metadata-driven automated construction and update process, effectively solving the problems of poor flexibility, low performance, and high maintenance costs in traditional solutions. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a data query method provided in this application embodiment; Figure 2 This is a structural diagram of a data query device provided in an embodiment of this application. Detailed Implementation

[0019] As described earlier, current data query systems generally suffer from fragmented patterns, poor flexibility, and performance bottlenecks in metric acquisition. Mainstream solutions are polarized: visual BI tools are easy to use but only support fixed aggregations, making it difficult to implement complex business logic (such as nested calculations and conditional weighting); while SQL or script-based data warehouse tools, although expressive, have high barriers to entry, making them difficult for business personnel to use independently. Multi-table joins generally rely on real-time JOINs, resulting in severe query latency when dealing with large tables or high cardinality dimensions. Some platforms use wide table pre-joins to improve performance, but their structure is rigid; adding dimensions or adjusting granularity requires rebuilding the model and ETL, leading to high iteration costs and slow response times, making it difficult to meet the needs of agile analysis.

[0020] In view of the above problems, this application provides a data query generation method and related products. The method includes: obtaining indicator metadata through indicator patterns; the indicator patterns include visual configuration indicator patterns or formulaic definition indicator patterns; constructing a relational wide table based on the relational table information in the indicator metadata; synchronizing upstream table data updates based on the relational wide table to obtain relational data; constructing a materialized view based on the indicator metadata and the relational data; the materialized view is constructed based on the indicator metadata, the relational data, and pre-calculated results; the pre-calculated results are obtained by pre-calculating based on the indicator metadata and the relational data; matching a query request with the materialized view, and determining the pre-calculated result that matches the query request as the final result.

[0021] This application generates consistent indicator metadata through two indicator modes: unified visual configuration and formulaic definition. This ensures logical alignment and consistent results between business and technical personnel in indicator definitions. Based on the related table information in the metadata, a wide related table is pre-built and dynamically updated to avoid high-overhead real-time JOINs during queries. Furthermore, by combining indicator metadata, related data, and pre-calculated results, a materialized view is automatically constructed, encapsulating business semantic aggregation logic. When a user queries, the system intelligently matches the materialized view and directly returns the pre-calculated results, eliminating redundant calculations. This application significantly reduces latency and resource consumption, improves concurrency performance, and achieves an end-to-end closed loop from indicator definition to accelerated queries through a metadata-driven automated process, effectively overcoming the pain points of traditional systems such as poor flexibility, low performance, and high maintenance costs.

[0022] Explanation of relevant professional terms: Dual-mode drive: It supports both "visual configuration" and "formulaic definition" indicator creation modes. The two modes can be switched seamlessly and the calculation results are guaranteed to be equivalent. Visual configuration is for business users (drag and drop operation), while formulaic definition is for data experts (custom expressions).

[0023] Pre-computation: Through technologies such as pre-aggregation, materialized views, and wide table construction, operations such as indicator calculation, table association, and data filtering are completed in advance and the results are stored. During querying, the pre-computed data can be directly reused to reduce the real-time computing pressure. The core includes capabilities such as pre-aggregation, filter awareness, and dimension roll-up.

[0024] Materialized view: The physical storage carrier of pre-computed results. It stores the pre-aggregated and associated datasets in a table structure, supporting direct query hits and incremental updates.

[0025] Derivative indicators: Indicators generated based on basic indicators through time rules (year-on-year and month-on-month comparisons, cumulative time, beginning and end of period) or inter-row calculations, such as "sales revenue in the same period last year" and "cumulative sales volume from the beginning of the month to date".

[0026] Composite metrics: Metrics generated through arithmetic operations (addition, subtraction, multiplication, and division) on multiple basic metrics, such as "Average order value = Sales revenue / Order quantity" and "Remaining inventory = Sales quantity - Inventory quantity".

[0027] Query rewriting: The system automatically parses the user's query logic, matches it with existing materialized views, and converts the original query statement into a statement that calls pre-calculated results, thereby accelerating the query process.

[0028] Filter awareness: Dimensional filter condition information is retained during the pre-calculation process. If the filter condition is a subset of the pre-calculated filter condition during the query, the materialized view can be directly hit, improving the hit rate.

[0029] Dimension roll-up: Supports aggregated calculations of metrics across different time granularities (year / month / day / hour) or dimension levels (province / city / district / county), such as quickly generating "monthly sales" based on the pre-calculated results of "daily sales".

[0030] Row-level permissions: Restrict data access scope based on user roles or attributes, allowing users to query row data only within their own permission scope, such as sales personnel only viewing sales data for their own region.

[0031] Wide table mode: The fact table and dimension table are joined in advance and integrated into a single redundant data table, avoiding real-time joins during queries and improving query speed.

[0032] def formulas: User-defined expression language (similar to Excel formulas) that supports complex calculation logic, such as def(sum_agg(sales amount), province) to calculate sales amount for each province.

[0033] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0034] Figure 1 A flowchart of a data query method provided in an embodiment of this application is shown below. Figure 1 As shown, a data query method includes: S101: Obtain indicator metadata through indicator patterns.

[0035] As described, the indicator modes include a visual configuration mode and a formula-based definition mode. This application supports users initiating requests through either visual drag-and-drop or formula writing during the indicator creation process. It also innovatively designs a dual-engine collaborative architecture: on the one hand, it provides a graphical configuration interface for business users, supporting quick indicator definition through dimension selection, aggregation method setting, and condition filtering; on the other hand, it provides data experts with expression-based formula definition capabilities, supporting complex scenarios such as nested functions, conditional logic, and cross-period calculations. The two modes achieve bidirectional mapping and semantic alignment through a set of formal equivalence transformation rules, ensuring that regardless of the definition method used, the final output standardized indicator metadata (including precise calculation logic, related table dependencies, dimension / indicator fields, data types, and business definitions) is completely consistent.

[0036] It comprehensively covers the needs of all scenarios for basic, derived, and composite indicators, ensuring flexibility while eliminating the risk of "different meanings for the same name." Rigorous verification shows that the numerical error of the calculation results generated by the two modes does not exceed 0.01%, effectively avoiding data discrepancies caused by mode switching and ensuring the accuracy and reliability of the indicators.

[0037] S102: Construct a wide relational table based on the relational table information in the indicator metadata.

[0038] Based on the explicit association table information in the indicator metadata, a wide association table supporting 1:1 and 1:N relationships is constructed in advance. The fact table and dimension table required for high-frequency queries are pre-associated during the data preparation stage, forming a reusable and clearly structured association data carrier, effectively avoiding the high latency caused by real-time JOIN during queries.

[0039] S103: Based on the synchronous update of the upstream table data of the associated wide table, the associated data is obtained.

[0040] This application does not limit the method for obtaining related data. For example, Change Data Capture (CDC) technology (such as based on database log parsing, triggers, or timestamp fields) can be used to monitor data changes (including add, modify, and delete operations) in the upstream fact table and dimension table in real time, and incrementally update the corresponding records in the related wide table; alternatively, incremental data can be pulled based on fields such as version number and update time through scheduled tasks (such as daily / hourly batch processing), and merging or full reconstruction strategies can be executed; in addition, low-latency dynamic refreshing of wide tables can be achieved by combining streaming processing engines (such as Apache Flink and Spark Streaming).

[0041] In 1:N join scenarios, sub-records can be automatically expanded or join results aggregated to ensure that the wide table structure adapts to query requirements. It also supports consistency verification mechanisms (such as checksum and row count comparison) and rollback strategies to prevent data distortion due to synchronization anomalies. Regardless of the method used, the final output join data remains logically consistent with the source table, is timely and reliable, and serves as a high-quality input foundation for subsequent materialized view construction and pre-computation.

[0042] To balance performance and governance, this application adopts a hybrid approach of "wide table construction + dimensional modeling." On the one hand, pre-JOIN improves query efficiency; on the other hand, it retains independent storage and unified management of dimension tables to ensure master data consistency and business semantic compliance. A built-in automated JOIN logic engine automatically identifies the join type (1:1 or 1:N) based on metadata, intelligently generates the optimal join strategy, and introduces mechanisms such as deduplication, null value handling, and foreign key validation during wide table construction to avoid data redundancy or bloat caused by improper joins. Simultaneously, by monitoring data changes in the upstream fact table and dimension tables (such as CDC logs or scheduled snapshots), the wide table content is periodically incrementally synchronized and refreshed to ensure high consistency with the source tables. This optimized JOIN logic not only improves construction efficiency but also guarantees the accuracy and integrity of the wide table data from the source, achieving an organic unity of high performance and strong consistency.

[0043] S104: Construct a materialized view based on the indicator metadata and the associated data.

[0044] After receiving metric metadata and associated data, a materialized view and corresponding storage structure adapted to business needs can be intelligently generated. This application does not limit the specific form of the materialized view. For example, the materialized view is not statically predefined, but dynamically constructed based on metric metadata (such as calculation logic, dimension / metric definitions, and filtering conditions), associated data characteristics (such as data size, update frequency, and distribution skew), and pre-calculated results. In specific applications, the "pre-aggregation + filter awareness + dimension roll-up + query rewriting" technology can be integrated to intelligently generate materialized views based on query history and data characteristics. During queries, the rewriting logic matches the pre-calculated results, fundamentally solving the hit rate and performance issues.

[0045] Specifically, based on historical query logs and current indicator definitions, the system automatically analyzes high-frequency access patterns (such as frequently searched "regional sales total in the last 30 days") and identifies complex computational bottlenecks (such as multi-level nested aggregation and cross-period comparison). This allows for intelligent decision-making regarding the core parameters of the materialized view: including the dimensions and indicator combinations to be included, typical filtering conditions (such as time range and status identifiers), optimal storage granularity (such as aggregation by day and aggregation by province), and a reasonable update cycle (such as hourly incremental refresh and daily full reconstruction). Based on this, pre-calculation tasks are automatically generated and submitted to the scheduling engine for execution, ensuring that the materialized content covers hot queries while avoiding redundant calculations. This mechanism represents an evolution from "human experience-driven" to "data and semantic dual-driven," significantly improving the hit rate, timeliness, and return on investment of materialized resources.

[0046] S105: Match the query request with the materialized view, and determine the pre-calculated result that matches the query request as the final result.

[0047] This application does not limit the matching process between query requests and materialized views. In one implementation, the query request submitted by the user is semantically parsed to extract its core logical elements, including aggregation functions, dimension fields, filtering conditions, and grouping granularity. Subsequently, based on filter-aware and dimension roll-up capabilities, it is intelligently determined whether there is a reusable materialized view. For example, when the query condition is "Q1 2024" and the materialized view contains "full year 2024" data, it can automatically roll up to a higher granularity and push down the filter; or when the query dimension is "province" and the materialized view is stored by "city", dimension generalization is achieved through aggregation.

[0048] Based on this, the query logic is automatically rewritten, transforming it into an equivalent query to the target materialized view. The rewritten request is then efficiently matched against the registered materialized view metadata. If a match is found, the corresponding pre-calculated result is directly read from the materialized view as the final output, avoiding access to the underlying large table or repeated complex calculations.

[0049] The entire matching and rewriting process is highly optimized. Relying on a lightweight rule engine and index acceleration mechanism, the end-to-end latency is controlled within ≤50ms, with a negligible impact on the overall query response latency. This ensures that the pre-computation mechanism improves performance without becoming a system bottleneck. It achieves a transparent, efficient, and low-overhead query acceleration experience, allowing users to seamlessly enjoy the performance benefits of materialized views without modifying their existing SQL or analysis habits.

[0050] The above describes the main technical solution of this application. Further implementations of the main technical solution are now introduced. Details are as follows: Regarding S101 obtaining indicator metadata through indicator patterns, this application provides an optional embodiment: If the indicator mode is the visual configuration indicator mode, then develop a drag-and-drop interface and determine the drag-and-drop operation; quickly create indicators through the drag-and-drop operation to obtain indicator data.

[0051] The drag-and-drop interface presents selectable data fields (such as dimensions and measures), aggregate functions (such as summation, average, count, maximum, deduplication count, etc.), and filter condition configuration panels; users can create atomic indicators or simple composite indicators by dragging fields to the corresponding areas (such as "Dimension Area", "Indicator Area", "Filter Area").

[0052] The drag-and-drop interface is deeply adapted to the usage habits of business personnel, eliminating the need for coding: for example, dragging the "Sales Revenue" field into the indicator area and selecting "Sum," dragging "Region" into the dimension area, and adding a filter condition "Time = Last 30 Days" automatically generates the corresponding indicator logic. The backend synchronously parses the drag-and-drop operations, transforming them into structured indicator metadata (including calculation expressions, related tables, data types, etc.), ensuring strict alignment with the formulaic definition pattern in terms of semantics and results. This approach significantly lowers the barrier to entry, improves indicator creation efficiency, and provides a standardized input foundation for subsequent pre-calculation, materialized view construction, and query acceleration.

[0053] If the indicator pattern is the formula-defined indicator pattern, then develop a syntax parser to determine the formula logic writing data; based on the formula logic writing data, quickly create the indicator to obtain the indicator data.

[0054] The parser supports structured expression syntax (such as using the `def` keyword to define metric names and calculation logic), and is compatible with common arithmetic operations (addition, subtraction, multiplication, division, parenthesis precedence), logical judgments (IF / ELSE, CASE WHEN), aggregate functions (SUM, AVG, COUNT DISTINCT), and time-based intelligent functions (same-year comparison, month-on-month comparison, rolling window, cumulative summation, YTD / QTD, etc.), thereby meeting the needs for accurate modeling of derived metrics (such as "gross profit margin = (revenue - cost) / revenue") and composite metrics (such as "30-day active user retention rate") in complex business scenarios.

[0055] Users can directly input formulas through the script editor, for example: def revenue_yoy=(SUM(revenue)-SUM(revenue, period='last_year')) / SUM(revenue, period='last_year'); The syntax parser transforms the data into standardized indicator metadata, including dependent fields, related tables, computational expression trees, time context, and output types. This process not only preserves the expressiveness and flexibility of the formulas but also ensures that they are semantically consistent with the visualization configuration mode and that the results are comparable. The resulting indicator data can be seamlessly integrated into subsequent wide table construction, materialized view pre-computation, and query acceleration processes, empowering data experts to build models efficiently while ensuring the uniformity and maintainability of the entire indicator system.

[0056] To ensure that the logic of the indicators generated by the visual drag-and-drop interface is completely equivalent to the logic written in formulas in terms of semantics and calculation results, especially when dealing with complex time functions (such as month-on-month, rolling window, cumulative summation), nested expressions, or multi-layered composite indicators defined by def, the indicator data is standardized to obtain indicator metadata. The indicator metadata accurately describes the indicator's dependencies, calculation topology, time context, aggregation path, and filtering conditions in the form of Intermediate Representation (IR), serving as a "common language" for the conversion between the two types of modes.

[0057] Building upon this foundation, a seamless two-way conversion between visual configuration and formula definition is achieved: indicators constructed by users in the drag-and-drop interface can generate equivalent .def formulas in real time for expert review or reuse; conversely, input formulas can also be rendered into visual components, facilitating understanding and adjustment by business personnel. To support efficient pre-calculation, a dedicated pre-calculation model is designed for high-frequency and complex logic such as year-on-year and month-on-month comparisons, cumulative values, and cross-period comparisons. This model can automatically decompose sub-expressions in .def formulas, identify materializable intermediate results (such as base period values ​​and scrolling window aggregations), and generate the optimal materialized view strategy accordingly. This not only ensures the consistency and accuracy of indicators under different creation methods but also effectively accelerates complex logic, truly achieving a unification of "flexible definition" and "high-performance query."

[0058] This application does not limit the standardization process. As an optional embodiment, if the indicator data is determined by the visualization configuration indicator mode, the indicator data is converted into standardized calculation logic to obtain indicator metadata; if the indicator data is determined by the formula definition indicator mode, the indicator data is parsed into execution logic to obtain indicator metadata.

[0059] For example, standardizing the indicator data based on the Abstract Syntax Tree (AST) (ensuring that the two types of patterns have the same format) generates indicator metadata (including calculation logic, association tables, dimension / indicator list, indicator type), enabling bidirectional parsing of visualization logic and formula logic.

[0060] This application uses a dual-mode drive (i.e., indicator mode) to obtain indicator metadata, which is superior to the single mode in many aspects, as shown in Table 1: Table 1

[0061] Table 1 compares the key performance indicators of the traditional single-mode data query system and the dual-mode driven architecture proposed in this application. Regarding indicator creation efficiency, the traditional model requires over one hour for complex indicators, while this application, through the collaboration of a visual drag-and-drop interface and a formula writing engine, compresses the creation time of complex indicators to within 10 minutes, significantly improving development efficiency. In terms of multi-table query latency (tens of millions of data points), traditional solutions relying on real-time JOIN have latency as high as 10-30 seconds, while this application, through pre-associated wide tables and materialized views, controls latency to within 3 seconds, greatly optimizing the user experience. Regarding pre-calculated hit rate, the traditional manual configuration method has a hit rate of less than 50%, while this application, based on intelligent matching and query awareness mechanisms, improves the hit rate to over 80%. For access-sensitive users, the traditional solution has a hit rate of only about 30%, while this application, through permission-embedded matching, achieves a hit rate of over 75%. Finally, in terms of support for complex indicators, traditional systems have limited support for composite logic (≤50%), while this application supports over 95% of complex indicators (including nested, time functions, etc.), comprehensively covering business analysis needs. Overall, this application demonstrates its comprehensive advantages in efficiency, performance, accuracy, and flexibility.

[0062] Regarding access restrictions, this application provides an optional embodiment: Obtain permission rules.

[0063] This application does not limit the permission rules. For example, permission rules include information such as user roles and row-level permission scope (such as region, department).

[0064] The permission rules are embedded into the materialized view.

[0065] This application does not limit the specific implementation method of embedding permission rules into materialized views. In a preferred embodiment, fine-grained permission rules (such as "User A can only view data in this department" or "Role B cannot access sensitive fields") are dynamically converted into structured filtering conditions (e.g., dept_idIN(101, 102) or sensitive_flag=false), and integrated into the definition logic of the materialized view to generate a materialized view with permission flags. The materialized view completes data pruning during the pre-computation stage, ensuring that the query results naturally conform to the access control policy.

[0066] To avoid a surge in the number of materialized views due to permission combination explosion, a permission generalization and sharing optimization strategy is adopted: for user groups with the same data range or overlapping permission paths, a common view is generated by merging them; for high-dimensional permission dimensions, a bitmap index or RBAC (role-based access control) abstraction layer is introduced to build materialized views by role or organizational unit, rather than generating them per user. Thus, without significantly increasing storage overhead, complex permission scenarios involving multiple tenants, multiple roles, and cross-organizations are efficiently supported.

[0067] In practice, permission filtering conditions are deeply integrated into the pre-computation task scheduling process, participating in the automatic construction of materialized views along with indicator logic, time granularity, and dimension hierarchy. The generated permission-enabled materialized views can be persistently stored in high-performance analytical databases (such as Huawei Cloud DWS, StarRocks, ClickHouse, etc.), and automatically associated with corresponding views during the query matching phase using permission context (such as user tokens and role IDs). This not only ensures data security and compliance but also moves permission checks from "runtime filtering" to the "pre-computation phase," further improving query performance and system scalability.

[0068] Regarding S105, which matches the query request with the materialized view and determines the pre-calculated result matching the query request as the final result, this application provides an optional embodiment: Query permissions are determined based on the query request (including query metrics, filtering conditions, and display dimensions).

[0069] The query permissions are verified to obtain the verification result and permission identifier.

[0070] If the verification result passes, materialized view matching is performed based on the permission identifier, and the materialized view that matches the permission identifier is taken as the target materialized view.

[0071] During the query execution phase, real-time permission verification is performed based on query permissions (such as user ID, role, organizational unit, or access token), parsing the permission context to which it belongs, and accurately matching the pre-built materialized view with the corresponding permission identifier. This application transforms the traditional post-security model of "querying all data first and then filtering permissions" into a pre-security acceleration paradigm of "pre-pruning by permission and directly hitting the result".

[0072] By deeply integrating permission rules into the construction and registration process of materialized views, it is ensured that each materialized view contains only a subset of data accessible to authorized users. Queries do not require additional row-level or column-level filtering, and scans of the original large tables are avoided. This allows for the direct reuse of pre-computed results while ensuring data security and compliance, significantly reducing query latency and computational overhead. This "permission-aware + pre-computed integration" design fundamentally resolves the inherent conflict between data security and query performance, eliminating the risk of unauthorized access and achieving sub-second response times. It is particularly suitable for scenarios with stringent requirements for both security and efficiency, such as multi-tenant SaaS platforms, financial risk control, and government data sharing.

[0073] The query request is parsed to extract the query logic.

[0074] Considering the diverse forms and complex structures of query statements in actual business operations, this application further develops highly compatible SQL parsing and intelligent matching logic, which can automatically identify the core semantics in the user's original query and transparently rewrite it into an efficient call statement for materialized views. For example, based on the SPJG (Select-Projection-Join-GroupBy) relational algebra model, the query request is formally decomposed: the selection conditions, projection fields, join structures, and grouping / aggregation logic of the query request are extracted to construct standardized query logic.

[0075] If a match is successful, an equivalent materialized view query is automatically generated, replacing the original operations on the underlying large table. It not only supports standard SQL but is also compatible with complex nested queries, window functions, and multi-level subqueries generated by common BI tools, achieving seamless acceleration while maintaining semantic consistency. By basing materialized view matching on the theoretical foundation of relational algebra, it significantly improves the accuracy, robustness, and scalability of query rewriting, providing unified and efficient acceleration capabilities for diverse analysis scenarios.

[0076] If the query filtering conditions in the query logic are a subset of the filtering conditions in the target materialized view and the query granularity in the query logic satisfies the roll-up generation conditions of the target materialized view, then the pre-calculated result of the corresponding target materialized view is extracted as the final result.

[0077] In practical applications, time-sensitive indicators such as year-on-year and month-on-month comparisons, cumulative values, and composite indicators composed of multiple basic indicators often rely on flexible roll-up of the time dimension (such as aggregating from "day" to "month") or collaborative pre-computation between multiple indicators. To efficiently support such scenarios, this application proposes that when constructing materialized views, it not only pre-computes fine-grained data (such as by day or by city), but also explicitly retains the original dimension filtering conditions and time context, and designs a storage structure and computational logic that supports multiple granular levels (year / quarter / month / day / hour) and dimension levels (such as country → province → city).

[0078] During the query phase, the system dynamically determines whether existing pre-calculated results can be reused by assessing the subset relationship between user filtering conditions and materialized view generation conditions (e.g., whether a user query for "Q1 2024" can be derived from a materialized view that includes "full year 2024"; or whether "East China" can be derived from the aggregation of "Shanghai + Jiangsu + Zhejiang"). This ensures that coarse-grained queries do not require recalculation but are generated in real-time based on existing fine-grained materialized data, guaranteeing result accuracy while avoiding redundant storage.

[0079] By incorporating filtering conditions into the metadata modeling of materialized views and combining the roll-up capabilities of dimensions and time, this application significantly improves the generalization and matching flexibility of pre-computation results, greatly increasing the hit rate of materialized views. It fundamentally solves the core pain points of traditional solutions, such as "narrow pre-computation coverage, low hit rate, and serious waste of resources," and achieves high-performance, highly reusable intelligent acceleration while ensuring the expressive power of complex indicators.

[0080] To address the issue of failure to match the materialized view, this application provides an optional embodiment: If no pre-calculated result is found to match the query request, the corresponding associated wide table and indicator metadata are called based on the query request to perform real-time calculation and obtain the final result.

[0081] If no pre-calculated result corresponding to the query request is found (i.e., no applicable materialized view), the system automatically falls back to the real-time calculation path: based on the dimensions, metrics, and filtering conditions parsed from the query request, combined with the already constructed related wide table and standardized metric metadata, an execution plan is dynamically generated and calculated in real time. Reusing the pre-related wide table structure avoids the temporary JOIN overhead of multiple source tables. Simultaneously, relying on the precise calculation logic defined in the metric metadata (such as aggregation methods, time functions, and business definitions), it ensures that the real-time calculation results are completely consistent with the pre-calculated results in both semantics and numerical value. This application balances query coverage with performance and accuracy, achieving a seamless experience of "accelerating where possible, and reliably backing up where acceleration is not possible."

[0082] For scenarios where the materialized view is updated in real time, this application provides an optional embodiment: Get updated and adjusted data in real time.

[0083] This application does not limit the data to be updated or adjusted. As mentioned above, the data to be updated or adjusted shall include at least one of the following: permission rule update data, indicator metadata update data, and related wide table structure update data.

[0084] The materialized view is updated in real time based on the updated adjustment data.

[0085] This application does not limit the update mechanism for materialized views and related components, supporting flexible, efficient, and consistent full lifecycle management. At the data level, log-based Change Data Capture (CDC) technology can be used to synchronize only the data that has actually changed in the upstream tables, significantly reducing transmission and computational overhead. Simultaneously, it supports multiple methods such as API triggering, scheduled operations, or manual intervention, combining data update frequency and business timeliness requirements to dynamically select full or incremental update strategies, automatically determining the change type: if it is a local data change, a lightweight incremental refresh is performed; if it involves significant structural or logical adjustments, a full reconstruction is triggered to ensure the accuracy of the results.

[0086] Regarding metadata and logical evolution, when the definition of an indicator changes (such as formula adjustment or addition / reduction of dimensions), the old version of the indicator metadata and its associated materialized view are automatically archived, preserving the historical version completely and supporting on-demand backtracking and auditing; when user permissions change, the permission identifier and embedded filter conditions of the materialized view are updated in real time to ensure that the appropriate pre-calculated results can still be accurately matched after the permission change, avoiding query failures or security vulnerabilities due to permission mismatch.

[0087] Furthermore, in the face of changes to the basic table structure (such as adding or deleting fields, or adjusting data types) or iterative iterations of metric logic, the solution updates metric metadata, related wide table structures, and materialized view definitions in a coordinated manner, forming an end-to-end consistency guarantee loop. By deeply integrating data change awareness, update strategy scheduling, and version control, this solution effectively balances system resource consumption and business response efficiency while ensuring high data freshness and strong consistency, achieving an intelligent, reliable, and maintainable dynamic update system.

[0088] To address the issue of periodically cleaning up materialized views, this application provides an optional embodiment: Periodically obtain usage data for materialized views.

[0089] This application does not limit the data used, but the data used may include the number of hits.

[0090] The materialized view is cleaned based on the usage data.

[0091] In practical implementation, a full lifecycle governance mechanism for materialized views is established. By regularly analyzing their hit efficiency, storage usage, computational overhead, and query acceleration effects, materialization strategies are dynamically optimized. On the one hand, inefficient or invalid materialized views are automatically identified and cleaned up, such as views that have not been hit by queries for a long time, whose data has expired, or views with extremely low hit frequency but high storage / computation costs, thus releasing valuable storage and computing resources in a timely manner. On the other hand, historical materialized data that has exceeded the preset retention period is archived in layers (e.g., moved to cold storage) to avoid consuming high-cost real-time query resources.

[0092] It can also introduce a materialized view ROI (Return on Investment) evaluation model to quantify the ratio between the performance benefits (such as the reduction in average query latency and CPU savings) brought by each materialized view and its resource costs (storage space, refresh frequency, ETL time). Based on this metric, it automatically executes creation, retention, degradation, or eviction decisions to ensure that pre-calculated investments are always focused on high-value scenarios. At the same time, it supports the supplementation and logical correction of historical materialized data to address operational needs such as data repair and caliber adjustments.

[0093] This application effectively solves the resource waste problem of "blind pre-calculation and only increasing without decreasing" in traditional solutions, and truly realizes on-demand materialization, intelligent preservation and cost control. It is consistent with the direction of advanced industry practices (such as the "materialized view ROI-driven optimization" emphasized by competitors), but further improves the sustainability and economy of the system through more granular cost-benefit perception and automated closed-loop management.

[0094] In specific implementations, this application can also perform operation log recording operations. This application provides an optional embodiment: It comprehensively records the creation and update logs of materialized views, query hit logs, and access permission logs, forming a complete audit chain covering the data lifecycle, query behavior, and security controls. All logs include timestamps, operation subjects, involved objects (such as materialized view ID, user roles), operation types, and key parameters, ensuring that every operation is traceable, reproducible, and verifiable. It also supports efficient log querying and compliance auditing through a unified interface, outputting structured, standardized, and traceable log data.

[0095] To balance performance and long-term analysis needs, historical log data is automatically archived periodically to low-cost storage (such as object storage or a log repository) to avoid burdening online services. Based on the archived logs, in-depth analysis can be performed on query hit trends (such as high-frequency / low-frequency materialized view distribution, time-period hotspots), pre-calculated resource consumption (such as refresh frequency, storage growth, and computational overhead), and access permission patterns (such as role usage distribution and unauthorized access attempts). This provides data-driven decision-making support for materialization strategy optimization, capacity planning, security policy adjustments, and ROI evaluation. This mechanism not only meets enterprise-level auditing and compliance requirements but also transforms logs from "passive recording" into a "proactive optimization engine," supporting continuous system evolution and intelligent autonomy.

[0096] In practice, multiple fine-grained control rules can be configured to achieve a dynamic balance between resources, performance, and business needs. For example: Set a threshold for the total storage capacity of materialized views. When the pre-computed data usage approaches the upper limit, an inefficient view cleanup or archiving strategy will be automatically triggered to prevent storage resources from being exhausted.

[0097] Define a threshold for the intelligently generated query frequency (e.g., "queried ≥ 5 times in the past 7 days"), and automatically create materialized views only for query patterns that meet the popularity criteria, avoiding wasting resources on cold queries.

[0098] Define the scope of the dimensional roll-up hierarchy (e.g., support roll-up step by step from year to quarter to month to day to hour, but prohibit skipping across multiple levels) to ensure that the roll-up results are semantically reasonable and computationally controllable.

[0099] Configure the number of retries and backoff strategies for failed updates (e.g., a maximum of 3 retries with exponential backoff at intervals) to ensure eventual data consistency while avoiding task blocking or resource idleness due to temporary failures.

[0100] These rules can be flexibly adjusted according to business scenarios to form a configurable, observable, and interventionable governance framework, ensuring that the materialized view system remains efficient, stable, and controllable while operating automatically.

[0101] In practice, the system can automatically build and record the complete data lineage between metrics, wide tables, materialized views, and base tables, forming an end-to-end lineage graph. When metric data anomalies occur (such as numerical mutations, logical deviations, or missing data), users can quickly trace back to the source base table through the lineage link to accurately locate the problem link. Whether it is an underlying data quality issue, ETL logic error, wide table synchronization delay, or materialized view definition deviation, it can be investigated efficiently.

[0102] This mechanism not only supports field-level dependency tracking (e.g., which table and field a metric ultimately originates from), but also associates the calculation logic version, update timestamp, and responsible person information, achieving auditability across the entire "data-logic-personnel" framework. This fully meets enterprises' core demands for data governance, compliance supervision, and trusted analysis, significantly improving the transparency and reliability of data assets. As industry practice emphasizes, "data governance and problem identification" are key capabilities of modern indicator platforms. This application, through its built-in lineage tracing system, transforms post-event error correction into a closed-loop data quality management system that is controllable beforehand, observable during the process, and traceable afterward.

[0103] In practical implementation, the system can automatically build and persistently record the end-to-end data lineage between metrics, wide tables, materialized views, and base tables, forming a fine-grained, traceable lineage graph. When metric data anomalies occur (such as numerical deviations, sudden increases in null values, or logical inconsistencies), users can drill down layer by layer through the lineage link to quickly locate the root cause of the problem. Whether it is a data quality issue in the underlying base table, a wide table synchronization delay, a materialized view refresh failure, or a defect in the metric calculation logic itself, all can be accurately identified and diagnosed.

[0104] This lineage system not only covers table-level and field-level dependencies but also links indicator definition versions, calculation expressions, ETL tasks, and update timestamps, achieving end-to-end connectivity across "data—logic—task—personnel." This significantly enhances the transparency and credibility of data assets and more effectively supports enterprise-level data governance, compliance auditing, and troubleshooting needs. As industry practice emphasizes, "data governance and problem investigation" are among the core capabilities of modern intelligent indicator platforms. This application, through an embedded automated lineage tracing mechanism, transforms passive response into a proactive, traceable, controllable, and manageable data operation and maintenance closed loop, providing a solid foundation for highly reliable and trustworthy decision analysis.

[0105] In practical implementation, it supports the supplementation and correction of historical data for materialized views to address inconsistencies caused by changes in historical data of upstream base tables (such as data correction, caliber retrospection, or source system repair). When an update to a historical partition or record of a base table is detected, the materialized view can be automatically or on demand recalculated within a specified time range. Based on the corrected source data, the corresponding pre-calculated results are regenerated to ensure that the materialized view and the underlying data maintain strict consistency across the entire time dimension.

[0106] This application effectively addresses the pain point of traditional pre-calculation solutions where "once data is written, it cannot be retrospectively corrected," avoiding distortion of report results or deviations in business analysis due to adjustments in source data. By supporting precise location of the scope requiring backfilling based on time windows, data versions, or change identifiers, it ensures data consistency while balancing computational efficiency and resource consumption. As industry materials clearly indicate, "supporting the backfilling of historical data in materialized views" is one of the key requirements of enterprise-level indicator platforms; this application, through a built-in flexible, controllable, and auditable historical data synchronization mechanism, realizes the evolution of the pre-calculation system from "static snapshots" to "dynamic reliability," fully supporting analysis scenarios requiring high compliance and high accuracy.

[0107] This application also provides specific embodiments for different construction scenarios of materialized views: Traditional generation algorithms based on query frequency (prioritizing the creation of materialized views for high-frequency queries) focus on improving the performance of high-frequency simple queries and are suitable for fixed report scenarios.

[0108] Traditional computational complexity-based generation algorithms (prioritizing the creation of materialized views for complex calculations) focus on reducing the real-time computational pressure of complex queries and are suitable for self-service analysis scenarios; this application adopts a hybrid algorithm of "frequency + complexity" to take into account both types of scenarios.

[0109] Regarding the implementation of dimensional roll-up for materialized views, this application also provides specific embodiments: Traditional pre-computation generates multi-granularity materialized views (such as generating day, month, and year granularity simultaneously), which results in faster query speeds but consumes more storage.

[0110] Traditional queries rely on real-time roll-up of fine-grained materialized views (e.g., generating monthly granularity from daily granularity), which consumes little storage but requires a small amount of real-time computation. This application supports switching between two methods, automatically selecting the method based on the data update frequency (the latter is used for high-frequency updates, and the former is used for low-frequency updates).

[0111] This application applies to the financial industry (such as risk indicator monitoring (such as non-performing loan ratio, capital adequacy ratio), marketing effectiveness analysis (such as average order value, conversion rate)), the retail industry (such as sales data analysis (such as year-on-year and month-on-month sales, regional sales ranking), inventory management (such as remaining inventory, inventory turnover rate)), the manufacturing industry (such as production indicator monitoring (such as capacity utilization rate, qualification rate), cost analysis (such as unit product cost, energy consumption index)) and the government sector (such as public welfare data statistics (such as cumulative number of employed persons, social security participation rate)).

[0112] In its implementation, this application adopts a business-driven query rewriting architecture: core capabilities such as wide table construction, indicator logic parsing, materialized view matching, and query rewriting are all completed at the business-driven layer, rather than relying on the underlying database engine's JOIN optimizer or built-in materialized view functionality. Compared to the "engine rewriting" mode (i.e., the database automatically identifies and replaces the query plan), this application does not rely on the advanced features of a specific database and can run seamlessly on various storage engines (such as MySQL, StarRocks, Doris, ClickHouse, etc.), significantly improving system compatibility and deployment flexibility.

[0113] More importantly, because wide table construction and query rewriting are based on rich business semantics (such as indicator definitions, dimension levels, permission rules, and time-based intelligent logic), more accurate pre-computation matching and more complex logic pushdown can be achieved, breaking through the limitations of general database optimizers in semantic understanding. Although this approach increases the development and maintenance complexity of the middle platform layer, it gains independent control over the entire query acceleration process. It can adapt to the diverse data stacks of enterprises and support the efficient execution of high-level analysis scenarios (such as month-on-month comparisons, composite indicators, and permission-aware queries), truly realizing an intelligent query acceleration system that is "semantic-driven, cross-engine compatible, and has predictable performance."

[0114] This application supports flexible materialized view storage strategies: on the one hand, materialized views can be stored directly in existing business databases (such as MySQL, PostgreSQL, or enterprise data warehouses), making full use of existing infrastructure without the need to deploy additional dedicated storage components, significantly reducing the implementation threshold and operational complexity; on the other hand, high-performance dedicated OLAP engines (such as CubeStore, StarRocks, Doris, etc.) can also be selected as the storage backend, leveraging their columnar storage, vectorized execution, and efficient indexing capabilities to achieve better query performance and concurrent processing capabilities, especially suitable for large-scale, high real-time analysis scenarios.

[0115] Users can choose between cost-effectiveness and ultimate performance based on their own technology stack maturity, performance requirements, and resource investment. This design ensures rapid deployment in lightweight environments while meeting the stringent high-performance analysis needs of large systems, demonstrating good compatibility, scalability, and engineering practicality.

[0116] This application pre-builds dedicated materialized views for users with different permissions, embedding permission filtering logic into the pre-computation stage. This allows queries to directly target pre-trimmed authorized data, eliminating the need for dynamic filtering of sensitive or unauthorized content at runtime, thus significantly improving query response speed. Compared to traditional solutions where multiple users share the same materialized view and rely on row-level permission filtering during the query period, this application avoids redundant data scanning and post-filtering overhead, resulting in particularly outstanding performance advantages in high-concurrency or complex permission scenarios.

[0117] While this strategy may increase storage resources (due to the need to generate differentiated views for different permission combinations), the system intelligently manages this through fine-grained permission abstraction (such as aggregation based on roles or organizational units) and an ROI evaluation module: continuously monitoring the hit frequency, query acceleration benefits, and storage / computing costs of each materialized view, automatically eliminating inefficient views, and prioritizing the retention of high-value pre-computed results. Thus, while prioritizing query performance, it achieves dynamic optimization of storage costs and a reasonable balance between resource investment, taking into account security, efficiency, and economy.

[0118] This application achieves seamless collaboration between business personnel and data experts in metric creation through a dual-mode driven mechanism: business personnel can quickly generate atomic metrics by dragging and dropping through a visual interface, while data experts can write complex nested logic (such as def functions, year-on-year and month-on-month calculations, etc.) using formulaic definition patterns. The metric results generated by both methods are consistent, ensuring a balance between ease of use and flexibility. Specifically, the time for business personnel to create atomic metrics has been reduced from 30 minutes to 5 minutes, greatly improving work efficiency; data experts can also quickly create complex metrics, solving the problem of "single definition patterns" in traditional solutions.

[0119] To improve the performance of multi-table join queries, the system pre-builds wide tables to reuse frequently joined data, avoiding real-time JOIN operations. When processing tens of millions of data points, the response time for multi-table join queries has been reduced from 10 seconds to less than 3 seconds, significantly reducing query latency and ensuring data consistency. This successfully solves the problem of "high latency in table joins" and achieves the goal of "low-latency queries".

[0120] By combining intelligent materialized view generation technology with filtering perception, dimension roll-up and query rewriting strategies, the system achieves efficient matching of pre-computed results, increasing the materialized view hit rate from 50% to over 80% in existing technologies. Query performance in core scenarios is improved by 70%, effectively solving the problem of "inefficient pre-computed results" and achieving the goal of "high hit rate and high performance".

[0121] Permission rules are deeply integrated into the pre-computation process, ensuring that data access security is guaranteed without affecting the pre-computation hit rate. For users with row-level permissions, the materialized view hit rate reaches ≥75%, which not only meets strict security control requirements but also does not sacrifice query performance, solving the problem of "separation between permissions and pre-computation" and achieving "a balance between security and performance".

[0122] Furthermore, the system supports both incremental and triggered updates, enabling it to detect changes in upstream data and dynamically adjust the update frequency to balance data timeliness and resource consumption. Compared to full updates, incremental updates save approximately 60% of computing and storage resources, effectively avoiding data lag, solving the problem of "rigid update scheduling," and achieving the goal of "balancing timeliness and cost."

[0123] For derived indicators (such as month-on-month and cumulative values) and composite indicators (multi-indicator calculations), the system has designed a dedicated pre-calculation adaptation model that supports the pre-calculation needs of over 95% of complex indicators. This ensures that the query response time for complex indicators does not exceed 5 seconds, solving the problem of "insufficient support for complex indicators" and achieving the goal of "refined analysis adaptation." This not only improves the system's usability and flexibility but also achieves significant improvements in performance optimization, security, and resource management.

[0124] Figure 2 A structural diagram of a data query device provided in an embodiment of this application is shown below. Figure 2 As shown, based on the data query method provided in the preceding embodiments, this application also provides a data query device including: The dual-mode indicator definition module is used to obtain indicator metadata through indicator modes; the indicator modes include a visual configuration indicator mode or a formulaic definition indicator mode.

[0125] The table association management module is used to construct a wide association table based on the association table information in the indicator metadata; and to synchronize the upstream table data update based on the wide association table to obtain the association data.

[0126] The intelligent pre-calculation module is used to construct a materialized view based on the indicator metadata and the associated data; the materialized view is constructed based on the indicator metadata, the associated data, and the pre-calculation results; the pre-calculation results are obtained by pre-calculating based on the indicator metadata and the associated data.

[0127] The query execution module is used to match the query request with the materialized view and determine the pre-calculated result that matches the query request as the final result.

[0128] As an optional implementation, the dual-mode index definition module specifically includes: The visualization configuration submodule is used to develop a drag-and-drop interface and determine the drag-and-drop operation if the indicator mode is the visualization configuration indicator mode; the indicator is quickly created through the drag-and-drop operation to obtain indicator data.

[0129] The formula definition submodule is used to develop a syntax parser to determine the formula logic writing data if the indicator pattern is the formula definition indicator pattern; and to quickly create indicators based on the formula logic writing data to obtain indicator data.

[0130] The equivalent transformation submodule is used to standardize the indicator data to obtain indicator metadata.

[0131] As an optional embodiment, the equivalent transformation submodule is specifically used for: If the indicator data is determined through the visualization configuration indicator mode, then the indicator data is converted into standardized calculation logic to obtain indicator metadata.

[0132] If the indicator data is determined through the formulaic definition of the indicator pattern, then the indicator data is parsed into execution logic to obtain indicator metadata.

[0133] As an optional embodiment, the apparatus further includes: The permission embedding module is used to obtain permission rules and embed the permission rules into the materialized view.

[0134] As an optional embodiment, the query execution module is specifically used for: Query permissions are determined based on the query request.

[0135] The query permissions are verified to obtain the verification result and permission identifier.

[0136] If the verification result passes, materialized view matching is performed based on the permission identifier, and the materialized view that matches the permission identifier is taken as the target materialized view.

[0137] The query request is parsed to extract the query logic.

[0138] If the query filtering conditions in the query logic are a subset of the filtering conditions in the target materialized view and the query granularity in the query logic satisfies the roll-up generation conditions of the target materialized view, then the pre-calculated result of the corresponding target materialized view is extracted as the final result.

[0139] As an optional embodiment, the apparatus further includes: The query execution module is also used to perform real-time calculations based on the query request by calling the corresponding associated wide table and indicator metadata if no pre-calculated result is determined to match the query request, so as to obtain the final result.

[0140] As an optional embodiment, the apparatus further includes: The dynamic update scheduling module is used to acquire update and adjustment data in real time; the update and adjustment data includes at least one of the following: permission rule update data, indicator metadata update data, and associated wide table structure update data.

[0141] The materialized view is updated in real time based on the updated adjustment data.

[0142] As an optional embodiment, the apparatus further includes: An evaluation module is used to periodically acquire usage data of the materialized view; the usage data includes the number of hits; and the materialized view is cleaned based on the usage data.

[0143] This application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a data query method.

[0144] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a data query method.

[0145] This application provides a computer program product, including a computer program that, when executed by a processor, implements a data query method.

[0146] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and equipment embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and equipment embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0147] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data query method, characterized in that, The method includes: Indicator metadata is obtained through indicator patterns; the indicator patterns include visually configurable indicator patterns or formulaically defined indicator patterns. Construct a wide relational table based on the relational table information in the indicator metadata; Based on the synchronization of upstream table data updates using the aforementioned associated wide table, associated data is obtained; A materialized view is constructed based on the indicator metadata and the associated data; the materialized view is constructed based on the indicator metadata, the associated data, and the pre-calculated results; the pre-calculated results are obtained by pre-calculating based on the indicator metadata and the associated data. The query request is matched with the materialized view, and the pre-calculated result that matches the query request is determined as the final result.

2. The data query method according to claim 1, characterized in that, The process of obtaining indicator metadata through indicator patterns specifically includes: If the indicator mode is the visual configuration indicator mode, then develop a drag-and-drop interface and determine the drag operation; Quickly create metrics and obtain metric data using drag-and-drop operations; If the indicator pattern is the formula-defined indicator pattern, then develop a syntax parser to determine the formula logic writing data; Based on the aforementioned formula logic, data is written to quickly create indicators and obtain indicator data; The indicator data is standardized to obtain indicator metadata.

3. The data query method according to claim 2, characterized in that, The standardization process for the indicator data to obtain indicator metadata specifically includes: If the indicator data is determined through the visualization configuration indicator mode, then the indicator data is converted into standardized calculation logic to obtain indicator metadata; If the indicator data is determined through the formulaic definition of the indicator pattern, then the indicator data is parsed into execution logic to obtain indicator metadata.

4. The data query method according to claim 1, characterized in that, The method further includes: Rules for obtaining permissions; The permission rules are embedded into the materialized view.

5. The data query method according to claim 1, characterized in that, The step of matching the query request with the materialized view and determining the pre-calculated result that matches the query request as the final result specifically includes: Determine query permissions based on the query request; The query permissions are verified to obtain the verification result and permission identifier; If the verification result passes, materialized view matching is performed based on the permission identifier, and the materialized view that matches the permission identifier is taken as the target materialized view. The query request is parsed to extract the query logic. If the query filtering conditions in the query logic are a subset of the filtering conditions in the target materialized view and the query granularity in the query logic satisfies the roll-up generation conditions of the target materialized view, then the pre-calculated result of the corresponding target materialized view is extracted as the final result.

6. The data query method according to claim 1, characterized in that, The method further includes: If no pre-calculated result is found to match the query request, the corresponding associated wide table and indicator metadata are called based on the query request to perform real-time calculation and obtain the final result.

7. The data query method according to claim 1, characterized in that, The method further includes: Real-time acquisition of updated and adjusted data; the updated and adjusted data includes at least one of the following: permission rule update data, indicator metadata update data, and related wide table structure update data; The materialized view is updated in real time based on the updated adjustment data.

8. The data query method according to claim 1, characterized in that, The method further includes: Periodically obtain usage data for the materialized view; the usage data includes the number of hits. The materialized view is cleaned based on the usage data.

9. A data query device, characterized in that, The data query device includes: The dual-mode indicator definition module is used to obtain indicator metadata through indicator modes; the indicator modes include a visually configurable indicator mode or a formulaically defined indicator mode. The table association management module is used to construct a wide association table based on the association table information in the indicator metadata; and to synchronize the upstream table data update based on the wide association table to obtain the association data. The intelligent pre-calculation module is used to construct a materialized view based on the indicator metadata and the associated data; the materialized view is constructed based on the indicator metadata, associated data, and pre-calculation results; the pre-calculation results are obtained by pre-calculating based on the indicator metadata and the associated data. The query execution module is used to match the query request with the materialized view and determine the pre-calculated result that matches the query request as the final result.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the data query method according to any one of claims 1-8.

Images