A method and system for model construction and penetration details based on a table-based metadata-driven model

Abstract

The application discloses a model construction and penetration detail method and system based on a table pass metadata drive, and comprises the following steps: receiving theme model processing configuration information and penetration detail verification configuration information based on a table pass data system, converting the information into a table pass metadata object, and describing a data processing flow by using a directed acyclic graph; performing consistency verification on the table pass metadata object, solidifying configuration that passes the verification to generate a consistent execution basis; compiling a materialized refresh plan and reserving a permission injection position; executing the materialized refresh plan, filtering permissions according to a data permission constraint field, generating a result table and saving the result table; registering the result table as a data set for board visual query; when triggering detail verification, converting and generating detail query conditions according to a chart data point context and a preset summary-detail field mapping rule, and returning detail data. The application can solve the problem that there is a lack of traceable corresponding relationship between model construction results and detail data.

Description

Technical Field

[0001] This invention belongs to the field of computer data processing technology, specifically relating to a method and system for model construction and detail penetration based on metadata driven by a single table. Background Technology

[0002] With the development of regulatory data analysis and business intelligence technologies, building thematic data models based on the unified data system, combining them with visual dashboards to display data, and providing detailed transparency functions to support data verification and problem localization has become a common approach in the field of regulatory data application.

[0003] In practical applications, existing technologies are characterized by independent configurations for model processing, visualization, and detailed verification, making it difficult to maintain consistent data standards and ensuring traceability across the entire data chain. This makes it challenging to fully meet the data auditability requirements in regulatory scenarios. Furthermore, data processing efficiency, access control precision, and the degree of business-oriented reusability of thematic models remain insufficient, hindering rapid responses to changes in business needs. Summary of the Invention

[0004] Based on the aforementioned background and the problems existing in the prior art, this invention proposes a model construction and detail penetration method and system driven by a unified metadata framework. This system enables unified data standards and traceable execution results across the entire chain of thematic model construction, visualization, and detail verification, effectively improving data processing efficiency and ensuring compliance and auditability of data access. Another objective of this invention is to improve the collaborative efficiency of regulatory data modeling and display verification through unified metadata definition, consistency verification and solidification, controlled compilation and execution, pre-filtering of permissions, and summary-detail penetration transformation. This forms a complete business closed loop from thematic modeling, result materialization, dataset registration, dashboard display to detail penetration, reducing the maintenance costs of multi-module distributed configurations and adapting to the high reliability requirements of regulatory data analysis scenarios.

[0005] To achieve these objectives and other advantages of the present invention, a method and system for model building and detail penetration based on metadata driven by a single table are provided.

[0006] To achieve the above objectives, this invention provides a method for model construction and detail penetration driven by metadata in a single table, comprising the following steps: S1. Receive the subject model processing configuration information and the transparent detailed verification configuration information based on the One Table Data System, and convert the received configuration information into a One Table metadata object with stable identifier, dependency relationship and data permission constraint fields. The subject model processing configuration information uses a directed acyclic graph to describe the data processing flow. S2. Perform consistency verification on the metadata object of the table, and solidify the metadata configuration that passes the verification to generate an immutable consistency execution basis; S3. Based on the consistency execution basis, compile the one-table metadata object into an executable materialized refresh plan, and reserve a data permission injection position in the materialized refresh plan; S4. Based on the data permission constraint field, perform permission filtering on the execution result of the materialized refresh plan, generate the theme model result table, and save it; S5. Register the result table as a queryable dataset so that the dashboard display can use the dataset as a data source for visual querying; S6. When a detailed verification of the visual query results is triggered, the display conditions at the summary result level are converted into detailed query conditions based on the data point context information of the chart where the triggering operation is located and the preset summary-detail field mapping rules, and the query is executed to return detailed data.

[0007] Furthermore, the data system of the unified table includes at least a resource layer, an intermediate layer detailed dimension table, and an intermediate layer summary dimension table. The resource layer provides regulatory standardized detailed data. The intermediate layer detailed dimension table reconstructs the regulatory standardized detailed data of the resource layer according to the regulatory business theme to form detailed dimension data. The intermediate layer summary dimension table summarizes the intermediate layer detailed dimension table to form summary dimension data. The dependencies include upstream and downstream dependencies between processing nodes in the topic model, input field source relationships, and mapping relationships between summary dataset fields and detail dataset fields in the transparent detail dataset. The upstream and downstream dependencies include at least the dependencies of adapting the intermediate layer summary dimension table based on the intermediate layer detail dimension table generation mechanism.

[0008] Furthermore, the step of performing consistency verification on the metadata object of the table, and solidifying the verified metadata configuration into an immutable consistency execution basis, includes the following steps: Perform node uniqueness verification, endpoint validity verification, acyclicity verification, unique result node verification, node field reference validity verification, and data permission field legality verification on the directed acyclic graph in the aforementioned metadata object; After verification, the semantic configuration that affects the data calculation results is separated from the layout configuration that only affects the display effect. Then, normalization and hash calculation are performed. The hash value of the semantic configuration is used as a consistency identifier. The consistency identifier is associated and bound with the verification report, approval record, release version and running task to generate an immutable consistency execution basis.

[0009] Furthermore, the step of compiling the table-wide metadata object into an executable materialized refresh plan based on the consistency execution criteria includes the following steps: Based on the fixed configuration version identified by the consistency execution basis, the structured query language fragments corresponding to each processing node are recursively generated according to the dependency order of each processing node in the one-table metadata object. Based on the type of each processing node, corresponding query clauses or data definition statements are generated to form the calculation logic of each processing node; Based on the computational logic, storage operation statements are generated for the nodes marked as needing materialization in each of the processing nodes. The storage operation statements include table creation statements, time-period deletion statements, data insertion statements, and index creation statements. The materialization refresh plan includes the computational logic of each processing node and the storage operation statements of the nodes marked as needing materialization.

[0010] Furthermore, the step of filtering the execution results of the materialized refresh plan based on the data permission constraint field, generating the topic model result table, and saving it includes the following steps: Retrieve permission scope information in the current operation context; The permission scope information is converted into filtering conditions for the data permission constraint field, and the filtering conditions are injected into the reserved data permission scope injection position in the materialized refresh plan, so that the execution process of the materialized refresh plan only processes the data that is allowed to be accessed by the current operation context; Execute the materialization refresh plan, generate the result table corresponding to the theme model, and save it.

[0011] Furthermore, registering the result table as a queryable dataset includes the following steps: Register the result table as a queryable dataset, and configure field metadata, organization scope field, payment period field and available query condition set for the dataset, so that the dashboard display can provide users with filtering controls based on the available query condition set, and dynamically generate query conditions based on the filter values ​​selected by the user during the visual query. The "Organization Scope" field is used to adapt the data summarized in the "One Table Data System" according to regulatory standards to the internal entity level of the organization.

[0012] Furthermore, the step of converting the display conditions at the summary result level into detailed query conditions based on the data point context information of the chart where the triggering operation is located and the preset summary-detail field mapping rules includes the following steps: The data point context information includes at least the chart type, click dimension field and corresponding value, hierarchical position, and the chart's own inherited filtering conditions. Following the priority order that temporary penetration configuration is superior to component penetration configuration, and component penetration configuration is superior to indicator definition condition tree, the source conditions in the data point context information are merged with the preset summary-detail field mapping rules to generate parameterized filtering conditions for the target detail dataset as detail query conditions.

[0013] Furthermore, before compiling and generating the materialized refresh plan, the following steps are also included: The processing nodes are sorted according to the topological order of the processing nodes in the directed acyclic graph. When test run mode is enabled, in response to the instruction to start running from a specified node, only the specified node and its downstream nodes are run, and the associated downstream nodes that are not run are marked as expired.

[0014] Furthermore, the visual query results of the dashboard display stage establish a traceable correspondence with the result table of the topic model through the consistency execution criteria; When a detailed verification of the visualized query results is triggered, the version of the result table corresponding to the visualized query results and the version of the detailed dataset corresponding to the detailed data are located according to the consistency execution criteria.

[0015] A model building and detail penetration system based on single-table metadata is provided to implement a model building and detail penetration method based on single-table metadata. The system includes: The OneTable metadata construction module is used to receive the subject model processing configuration information and the transparent detailed verification configuration information based on the OneTable data system. It transforms the received configuration information into a OneTable metadata object with stable identifier, dependency relationship and data permission constraint fields. The subject model processing configuration information uses a directed acyclic graph to describe the data processing flow. The consistency verification and solidification module is used to perform consistency verification on the metadata object of the table and solidify the metadata configuration that passes the verification to generate an immutable consistency execution basis. The materialized refresh plan compilation module is used to compile the one-table-through metadata object into an executable materialized refresh plan based on the consistency execution basis, and to reserve injection positions for data permissions in the materialized refresh plan; The permission filtering and result materialization module is used to filter the execution result of the materialized refresh plan based on the data permission constraint field, generate the topic model result table and save it. The dataset registration and visualization query module is used to register the result table as a queryable dataset, so that the dashboard display can use the dataset as a data source for visualization query. The summary-detail penetration query module is used to convert the display conditions at the summary result level into detailed query conditions when the detailed verification of the visual query results is triggered, based on the data point context information of the chart where the triggering operation is located and the preset summary-detail field mapping rules, and then execute the query to return detailed data.

[0016] The present invention has at least the following beneficial effects: First, by connecting the model construction, visualization and detailed verification through a unified metadata system, and combining the consistent execution basis of semantic configuration hash solidification, the verification, approval, release and operation are bound to the same semantic benchmark. This avoids the problems of caliber drift and unreproducible results caused by the scattered configuration of multiple modules from the root, and fully meets the data auditability requirements of regulatory scenarios.

[0017] Secondly, topological sorting is adopted to ensure that data processing proceeds in an orderly manner along the dependency link. Combined with the intermediate result reuse mechanism of materialized nodes, the overhead of repeated calculations is reduced. Permission filtering is moved to the data reading stage to exclude invalid data scanning from the source. With the optimization of billing period partitioning and multi-dimensional indexes, the execution efficiency of data processing and query is effectively improved.

[0018] Meanwhile, by using data permission constraint fields throughout the entire chain, combined with the mechanism of pre-reserved injection positions during the compilation stage and dynamic injection of agency scope filtering at runtime, unauthorized access is restricted from the source of data processing. Query caching is also included in the permission scope dimension, realizing the isolation of cached results for users with different permissions, fully ensuring the compliance and security of data access, and adapting to the regulatory requirements for hierarchical access to data.

[0019] In addition, it supports the direct registration of the materialized results of thematic models as standardized datasets, reusing existing dataset query, dashboard configuration and transparent detail conversion capabilities, eliminating the need to develop separate display interfaces and detailed logic for each thematic model. It also supports flexible data adaptation from regulatory scope to the level of internal entities, forming a complete business chain of modeling, materialization, display and verification, effectively reducing business implementation costs and shortening the business iteration cycle of regulatory analysis scenarios.

[0020] Finally, the node-based architecture based on the directed acyclic graph can flexibly combine various data processing operations, supporting multiple execution modes such as direct query, extraction query, online pagination, and asynchronous export. It can adapt to the differentiated needs of different data scales and business scenarios under the unified data system, and has strong engineering implementation and expansion capabilities.

[0021] Other advantages, objectives and features of the present invention will become apparent in part from the following description, and in part from those skilled in the art through study and practice of the invention. Attached Figure Description

[0022] Figure 1 This is a schematic flowchart of the method of the present invention; Figure 2 This is a functional block diagram of the system of the present invention. Detailed Implementation

[0023] To clearly illustrate the present invention and make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, so that those skilled in the art can implement the invention based on the description. The technology of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] Example 1: A model construction and detail-oriented method based on single-table metadata, such as... Figure 1 As shown, it includes the following steps: S1. Receive the subject model processing configuration information and the transparent detailed verification configuration information based on the One Table Data System, and convert the received configuration information into a One Table metadata object with stable identifier, dependency relationship and data permission constraint fields. The subject model processing configuration information uses a directed acyclic graph to describe the data processing flow. S2. Perform consistency verification on the metadata object of the table, and solidify the metadata configuration that passes the verification to generate an immutable consistency execution basis; S3. Based on the consistency execution basis, compile the one-table metadata object into an executable materialized refresh plan, and reserve a data permission injection position in the materialized refresh plan; S4. Based on the data permission constraint field, perform permission filtering on the execution result of the materialized refresh plan, generate the theme model result table, and save it; S5. Register the result table as a queryable dataset so that the dashboard display can use the dataset as a data source for visual querying; S6. When a detailed verification of the visual query results is triggered, the display conditions at the summary result level are converted into detailed query conditions based on the data point context information of the chart where the triggering operation is located and the preset summary-detail field mapping rules, and the query is executed to return detailed data.

[0025] The above steps follow a complete technical chain: structured definition, consistency verification, controlled compilation, scope constraint execution, result visualization, and penetration verification. Using a unified metadata table as the carrier, the entire process of model construction, access control, visualization, and detailed verification is connected. Consistency and traceability of execution are ensured through configuration solidification and version binding mechanisms. The implementation details of the above steps are explained below.

[0026] For step S1, the subject model processing configuration information and the transparent detailed verification configuration information based on the One-Table-Pass data system are received. The received configuration information is transformed into a One-Table-Pass metadata object with stable identifiers, dependency relationships, and data permission constraint fields. The subject model processing configuration information uses a directed acyclic graph to describe the data processing flow. The One-Table-Pass data system includes at least a resource layer, an intermediate layer detailed dimension table, and an intermediate layer summary dimension table. The resource layer provides standardized regulatory detailed data. The intermediate layer detailed dimension table reconstructs the standardized regulatory detailed data of the resource layer according to the regulatory business theme to form detailed dimension data. The intermediate layer summary dimension table summarizes the intermediate layer detailed dimension tables to form summary dimension data. The dependency relationships include: upstream and downstream dependency relationships between processing nodes in the subject model, input field source relationships, and mapping correspondence relationships between summary dataset fields and detailed dataset fields in the transparent detailed data system. The upstream and downstream dependency relationships include at least the dependency relationship between the intermediate layer detailed dimension table generation organization and the intermediate layer summary dimension table.

[0027] Specifically, the One-Table Data System adopts a hierarchical data architecture. The overall data flow is from the resource layer to the intermediate layer detailed dimension tables, then to the intermediate layer summary dimension tables, ultimately supporting various business applications at the application layer. The resource layer carries standardized raw detailed data formed according to regulatory standards, covering business areas such as institutions, customers, finance, loans, deposits, interbank transactions, investment, off-balance-sheet credit, employees, and financing. It retains a unified data structure and raw detail granularity, providing basic data support for upper-layer data processing and application. The intermediate layer detailed dimension tables reconstruct the standardized regulatory detailed data from the resource layer according to regulatory business themes, forming thematic detailed dimension data at the finest granularity of the business. For example, the loan dimension table organizes data at the loan document granularity, supporting drill-down from indicators to detailed data and providing a unified data foundation for indicator processing, risk models, and business analysis. The intermediate layer summary dimension tables perform light summarization of the intermediate layer detailed dimension tables, forming summary dimension data according to regulatory standards such as legal entities and administrative divisions, supporting statistical analysis at the regulatory level. Based on this, an organization-adapted intermediate-layer summary dimension table can be generated by processing the intermediate-layer detailed dimension table. The summary processing is carried out according to the hierarchical structure of the internal entities of the organization, which not only maintains the unified standard of regulatory data, but also meets the application needs of internal business analysis and risk management of the organization.

[0028] The application layer, based on the data from the intermediate layer's aggregated dimension tables and the organization's adapted intermediate layer aggregated dimension tables, uniformly consolidates data processing rules and application capabilities for business applications. This primarily includes five core capabilities: a unified indicator system, business analysis dashboards, risk verification models, model performance management, and autonomous applications. The unified indicator system generates indicator processing rules and results; the business analysis dashboards support analysis of organizational operations and key areas; the risk verification models construct risk identification and verification rules based on thematic data; model performance management generates results on the identification of model anomalies and the analysis of model effectiveness; and autonomous applications support flexible configuration of indicator dashboards, reports, and models.

[0029] The system also integrates data submitted through the 1104 reporting system and conducts cross-system data quality verification with the data submitted through the "One-Form" system. By establishing a correspondence between the "One-Form" system indicators and the 1104 indicators, comparative analysis is performed on indicators with the same caliber. Further verification is conducted by combining the detailed dimension table data from the intermediate layer of the "One-Form" system with the local 1104 reporting details of the institution. This achieves a complete verification chain for indicator difference discovery, difference analysis, detailed comparison, and problem localization, resolving potential differences in indicator caliber, data inconsistencies, and source data quality issues between different regulatory reporting systems.

[0030] After receiving the processing configuration of the topic model, the core elements such as source table definitions, processing operation rules, and output targets are first parsed. Each independent data processing operation is abstracted into a corresponding processing node. Each node carries attributes such as operation type, input / output field definitions, and processing parameters. The types of processing nodes cover all categories of data processing scenarios, including filtering, deduplication, association, merging, retrieving the latest data, field transformation, aggregation, temporary materialization, and result output. Simultaneously, according to the data processing flow logic, directed edges are established between processing nodes with upstream and downstream input / output relationships. The direction of the directed edges is consistent with the data flow direction, thus forming a directed acyclic graph (DAG) that can completely describe the entire data processing process. Using a DAG to structurally represent the data processing flow clearly defines the execution order and dependency boundaries of each processing step, avoiding ambiguity caused by natural language descriptions, and providing a clear structural basis for subsequent topology sorting, dependency verification, and compilation execution.

[0031] The configuration information for transparent detailed verification is synchronously incorporated into the processing scope. This includes parsing the target detailed dataset identifier, the correspondence rules between summary fields and detailed fields, the conditional mapping method, and the penetration trigger rules. This information is then integrated as penetration configuration attributes into a unified metadata system and linked to the result output nodes of the topic model. This achieves unified management of model processing configuration and detailed verification configuration. Integrating transparent detailed configuration and topic model processing configuration into the same metadata system avoids discrepancies caused by maintaining the two types of configurations separately, providing a unified semantic foundation for subsequent consistency verification of summary results and detailed data.

[0032] After the core elements have been structurally transformed, a globally unique and stable identifier is assigned to each processing node, directed edge, field definition, and all metadata elements, including penetration configuration items. These identifiers are generated using fixed encoding rules and do not change with non-semantic modifications such as layout adjustments or description changes. This stable identifier system ensures the recognizability of elements during configuration iterations, preventing element matching failures due to changes in description text or display position, and guaranteeing the traceability of configurations before and after version iterations.

[0033] At the dependency construction level, firstly, based on the edge connections of the directed acyclic graph, a set of upstream dependent nodes and a set of downstream successor nodes are constructed for each node. This clarifies the preconditions for execution and the subsequent scope of influence for each node, forming a complete node-level upstream and downstream dependency chain. Clear upstream and downstream dependencies support subsequent compilation and computation in topological order, avoiding logical errors caused by executing before dependencies are ready, and providing a structural basis for local recalculation and link fault location. Secondly, for each processing node's input field, its corresponding output field from its source node is traced one by one, establishing a field-level lineage mapping relationship. The flow path and transformation logic of each field are recorded, forming the input field source relationship of the processing node. Field-level source relationships enable full-link tracing of data lineage, facilitating the location of the source of field calculation errors and providing a data foundation for subsequent field reference validity verification. For transparent detail scenarios, based on transparent detail configuration, a mapping relationship is established between summary-side dimension fields, metric fields, and corresponding fields on the detail side. Simultaneously, mapping rules for conditional operators and default value inheritance methods are defined, forming a standardized mapping relationship between summary dataset fields and detail dataset fields. A clear field mapping relationship can accurately convert the filtering conditions and dimension values ​​at the summary level into detailed query conditions, avoiding problems such as field mismatch and condition loss during the penetration process, and ensuring that the statistical caliber of the summary results and detailed data is consistent.

[0034] In addition, the unified data permission constraint field is set in the metadata object of the unified table to carry the semantics of permission control, such as organizational scope, data visibility level, and role permissions. The permission constraint field can be associated with metadata elements at different levels, such as data sources, datasets, processing nodes, and output results, to achieve end-to-end permission attribute implementation. The unified data permission constraint field can embed permission control semantics into the metadata system, providing a unified control basis for permission filtering in the subsequent whole process and avoiding the problem of inconsistent permission standards caused by configuring permissions separately at each stage.

[0035] The above steps unify the scattered subject model processing configuration and the transparent detailed verification configuration into a single metadata object, realizing the standardization and structured representation of data processing flow, field lineage, penetration rules and permission constraints. This breaks down the barriers between the independent configurations of different modules and provides a unified data semantic benchmark for subsequent consistency verification, controlled compilation, range constraint execution and penetration verification, ensuring the consistency and traceability of the entire data scope from the configuration source.

[0036] For step S2, a consistency check is performed on the metadata object of the unified table. The metadata configuration that passes the check is solidified to generate an immutable consistency execution basis. This includes: performing node uniqueness check, endpoint validity check, acyclicity check, unique result node check, node field reference validity check, and data permission field legality check on the directed acyclic graph in the unified table metadata object. After the check passes, the semantic configuration that affects the data calculation result is separated from the layout configuration that only affects the display effect, and normalization processing and hash calculation are performed. The hash value of the semantic configuration is used as the consistency identifier, and the consistency identifier is associated and bound with the check report, approval record, release version, and running task to generate an immutable consistency execution basis.

[0037] Specifically, the node uniqueness check traverses the identifier field of all processing nodes in the directed acyclic graph to check for duplicate node identifiers. It also checks for conflicts in identifiable fields such as node names and technical aliases. This check ensures that each node has a unique identity globally, preventing ambiguity in node references and execution chain errors during subsequent compilation stages, thus guaranteeing the accuracy of compilation addressing.

[0038] The endpoint validity check iterates through all directed edges, verifying the existence of the start and end node identifiers of each edge in the defined node set, and excluding dangling edges whose start or end points point to non-existent nodes. This check ensures the integrity of data flow at the structural level, avoiding runtime interruptions caused by missing dependent nodes during subsequent execution and reducing the risk of process crashes.

[0039] The acyclicity check employs a depth-first search algorithm to traverse all paths in the directed acyclic graph (DAG), detecting whether a circular path originating from a node, follows a directed edge, and ultimately returns to the node. During the traversal, the visit status and recursion stack status of each node are marked. When a node already existing in the recursion stack is found, a cycle is determined to exist. This check ensures that the data processing flow possesses the topological property of directed acyclicity, guaranteeing the feasibility of subsequent execution in topological order and avoiding runtime failures such as infinite loops and infinite recursion caused by circular dependencies.

[0040] The unique result node verification counts the number of nodes with an out-degree of zero in the directed acyclic graph, and verifies whether the final output node count is 1. This verification ensures that the data processing chain has a unique convergence output point, avoids output ambiguity caused by multiple result nodes, and provides a clear output target for subsequent dataset registration and visualization calls.

[0041] The node field reference validity check verifies the existence of each field name in the output field set of all its direct upstream nodes for each input field list of each processing node, while also verifying the field type matching. This check proactively identifies issues such as incorrect field references and type mismatches at the configuration level, preventing runtime errors such as missing fields and failed data type conversions during execution. By intercepting faults at the configuration verification stage, it reduces the cost of troubleshooting runtime problems.

[0042] The data permission field validity check verifies whether the values ​​of data permission constraint fields on each metadata element conform to the predefined permission hierarchy coding rules, whether the associated organizational scope hierarchy structure is valid, and whether the scope of the permission field matches the data processing logic of the corresponding node. This check ensures that the format and values ​​of the permission constraint fields are valid and effective, avoiding problems such as condition parsing failure and permission control failure in the subsequent permission filtering stage, and ensuring the compliance foundation of data access.

[0043] After all validation items pass, the metadata configuration is categorized and split into two types: semantic configuration and layout configuration. Semantic configuration covers configuration items that directly determine the final data calculation result, such as the calculation logic of processing nodes, field mapping relationships, filtering conditions, permission constraints, and penetration mapping rules. Layout configuration covers configuration items that only affect the visual display effect and do not affect the data processing result, such as the coordinate position of nodes in the canvas, connection style, and display label format. Separating these two types of configuration decouples data processing semantics from display effects, providing a foundation for subsequent differentiated version management and iteration.

[0044] Subsequently, normalization processing was performed on both types of configurations. For semantic configurations, configuration items were sorted, deduplicated, and formatted according to a unified serialization rule, eliminating the problem of semantically identical but textually different items due to differences in spaces, line breaks, and word order. Layout configurations were also formatted normally, retaining display-related attributes. Normalization processing can eliminate interference from non-substantive differences in configuration consistency judgment and improve the accuracy of subsequent hash calculations.

[0045] After normalization, the semantic configuration hash value and the layout configuration hash value are calculated separately. The hash calculation uses the SHA-256 algorithm to calculate the hash value of the normalized configuration text, resulting in a fixed-length hash string, hash_value. The calculation formula is as follows: hash_value=SHA-256(normalized_config), Here, `normalized_config` represents the normalized configuration text. Based on the one-way and unique characteristics of hash algorithms, configurations with the same semantics will generate completely identical hash values. Any semantic change in the configuration will cause the hash value to change, which can accurately identify semantic changes in the configuration and provide an objective and unique identifier for determining the configuration version.

[0046] The hash value of the semantic configuration is used as a consistency identifier, which is then associated with and bound to the verification report, approval record, release version, and runtime task. All subsequent compilation, execution, and query operations are based on the configuration version corresponding to this consistency identifier. This binding mechanism ensures that the verification, approval, release, and runtime stages share the same semantic benchmark, thus avoiding the inconsistency problem of verifying the old configuration and running the new configuration, and ensuring the reproducibility of execution results. At the same time, the semantic configuration hash is separated from the layout configuration hash, so that when only the display layout is adjusted, the semantic configuration hash remains unchanged, without needing to re-trigger the verification and approval process, effectively improving the efficiency of configuration iteration and reducing unnecessary duplicate verification overhead.

[0047] The above steps proactively intercept various configuration defects through six dimensions of consistency verification. Then, through a hash-based solidification mechanism that separates semantics and layout, the verified configuration semantics are solidified and bound in the form of a consistency identifier, forming an immutable basis for consistent execution. This not only ensures the correctness and compliance of the configuration from the source, but also decouples the semantics of data processing from the display effect. It provides a core version benchmark for the consistency of the entire chain and the traceability of the results, and is the core link in the entire metadata-driven system that ensures the reproducibility of execution.

[0048] For step S3, based on the consistency execution basis, the unified metadata object is compiled into an executable materialized refresh plan, and a data permission injection position is reserved in the materialized refresh plan. This includes: based on the fixed configuration version identified by the consistency execution basis, recursively generating structured query language fragments corresponding to each processing node according to the dependency order of each processing node in the unified metadata object; generating corresponding query clauses or data definition statements according to the type of each processing node to form the calculation logic of each processing node; based on the calculation logic, generating storage operation statements for the nodes marked as needing materialization in each processing node, the storage operation statements including table creation statements, time period deletion statements, data insertion statements, and index creation statements; the materialized refresh plan includes the calculation logic of each processing node and the storage operation statements of the nodes marked as needing materialization. Furthermore, before compiling and generating the materialized refresh plan, the method further includes: sorting each processing node according to the topological order of each processing node in the directed acyclic graph; when the test run mode is enabled, in response to the instruction to start running from a specified node, only the specified node and its downstream nodes are run, and the associated downstream nodes that are not run are marked as expired.

[0049] Specifically, the compilation process first uses the fixed configuration version identified by the consistency execution basis as a benchmark, reading the corresponding version of the OneTable metadata configuration to ensure that the configuration used for compilation is a verified and fixed official version, not an editable draft, thus guaranteeing the consistency between execution logic and verification semantics from the compilation source. Based on this, following the dependency order of each processing node in the OneTable metadata object, starting from the upstream source node, it recursively traverses downstream along directed edges, generating structured query language fragments corresponding to each processing node layer by layer. Each node's query fragment is constructed based on the output results of all its direct upstream nodes, embedding the upstream node's query as a subquery into the current node's query logic, forming a nested query structure. Using a recursive approach to generate query fragments along the dependency chain strictly adheres to the upstream and downstream dependencies between nodes, avoiding dependency inversion errors, while ensuring a one-to-one correspondence between the query structure and the data processing flow, guaranteeing that the compiled execution logic is completely consistent with the business semantics defined in the configuration.

[0050] Based on the type of each processing node, corresponding query clauses or data definition statements are generated to form the calculation logic of each node. For filtering type nodes, a WHERE clause with corresponding filtering conditions is generated; for deduplication type nodes, a DISTINCT clause based on the deduplication field is generated; for association type nodes, a JOIN clause with corresponding association fields and conditions is generated; for aggregation type nodes, a GROUP BY clause with grouping dimension fields and aggregation functions, along with corresponding calculation expressions, is generated; for field transformation type nodes, a SELECT field list containing field transformation, renaming, and calculation logic is generated; for retrieve latest type nodes, a window function clause for grouping and sorting is generated to implement the logic of retrieving the latest record by group. Different statement generation rules for different node types can uniformly convert various data processing operations defined in the configuration into standard structured query language, achieving automated conversion from business configuration to executable code, avoiding syntax errors and logical deviations caused by manual coding, while ensuring strict alignment between calculation logic and configuration semantics.

[0051] Based on the computational logic of each node, all processing nodes marked as requiring materialization are identified, and corresponding storage operation statements are generated for them. The table creation statement generates the corresponding data table structure based on the node's output field definitions, ensuring consistency between field names, data types, and precision with the configuration definitions. The time-period deletion statement sets deletion conditions for the billing period field, deleting historical data for the corresponding billing period before each refresh execution to achieve idempotent refresh of data within the same billing period. The data insertion statement inserts the computation results of the current node into the generated materialized table, completing the persistence of computation results. The index creation statement creates indexes for the billing period field, organization field, and commonly used query dimension fields, optimizing the performance of subsequent reads and queries. The storage operation statements and node computational logic together constitute a complete materialization execution logic, enabling the persistence of node computation results to physical tables. Intermediate computation results can be reused by multiple downstream nodes, avoiding repeated calculations of the same intermediate results in long chains and effectively improving overall execution efficiency. The billing period deletion mechanism ensures the accuracy of data in each refresh, avoiding errors caused by mixing old and new data. Finally, the computational logic of all nodes and the storage operation statements of materialized nodes are integrated to form a complete materialized refresh plan. At the same time, the plan reserves injection positions for data permission ranges at the logical positions corresponding to data filtering, which are used to dynamically access permission filtering conditions during subsequent runtime, so that the same materialized refresh plan can adapt to the access needs of different permission subjects.

[0052] Before compiling and generating the materialized refresh plan, all processing nodes are first sorted according to the topological order of the nodes in the directed acyclic graph. The sorting is implemented using the Kahn algorithm based on node in-degree. First, the in-degree value in_degree(v) of each node is calculated using the following formula:

[0053] Where v is the target node whose in-degree is to be calculated, u is the upstream node with a directed edge pointing to v, E is the set of all directed edges in the directed acyclic graph, and count( The function `in_degree(v)` is used to count the number of elements in the set. Then, all source nodes with an in-degree of zero are added to the initial queue. Nodes are then sequentially removed and added to the sorting result. Simultaneously, all outgoing edges of the node are removed, and the in-degree of downstream nodes is updated. The in-degree update formula is: `in_degree(v) = in_degree(v) - 1`. When an edge exists...<u,v> Furthermore, when u has been removed from the queue, it is added back to the queue when the in-degree of a downstream node drops to zero. This process is repeated until all nodes are sorted. By using quantified in-degree calculation and iterative update rules, the execution order of each node can be accurately determined, avoiding dependency omissions or order errors caused by manual sorting. This ensures that subsequent execution processes proceed strictly according to the dependency relationships. Downstream node calculations are only started after upstream nodes have completed their calculations and output results, preventing downstream nodes from reading unread intermediate data. This ensures the correctness of the computational logic from the perspective of execution order.

[0054] When test run mode is enabled, in response to the instruction to start running from a specified node, the compilation scope is adjusted, compiling only the execution logic of the specified node and all its downstream nodes, generating a local materialized refresh plan. Simultaneously, related downstream nodes that depend on unrunnated upstream nodes are marked as expired, indicating that their result data is invalid. Test run mode can skip upstream nodes that do not require verification in debugging scenarios, running only the target node and its downstream links, significantly reducing the computational load during debugging, shortening the verification cycle after configuration adjustments, and improving development and debugging efficiency. The expired status marking clearly distinguishes between valid and invalid results, preventing downstream nodes from misusing outdated historical data and ensuring the reliability of results in debugging scenarios.

[0055] The above steps take the solidified consistency execution basis as input and automatically convert the structured business configuration into an executable materialized refresh plan through a dependency-driven recursive compilation mechanism. This achieves a precise mapping from configuration semantics to execution logic, ensuring the consistency and correctness of the computational logic. The accompanying topology sorting mechanism ensures the rationality of the execution order, and the test run mode takes into account the efficiency of debugging scenarios. At the same time, the reserved data permission injection position provides a standardized interface for subsequent range constraint execution, which is the core hub connecting configuration definition and actual data execution.

[0056] For step S4, based on the data permission constraint field, the execution result of the materialized refresh plan is filtered for permissions, and the topic model result table is generated and saved. This includes: obtaining permission range information in the current operation context; converting the permission range information into filtering conditions for the data permission constraint field, and injecting the filtering conditions into the reserved data permission range injection position in the materialized refresh plan, so that the execution process of the materialized refresh plan only processes the data that is allowed to be accessed by the current operation context; executing the materialized refresh plan, generating the result table corresponding to the topic model, and saving it.

[0057] Specifically, the acquisition of permission scope information corresponds to the triggering scenario of the current operation. For preview and query operations triggered by the user, the permission scope is derived from the organization code and corresponding data access level of the currently logged-in user, automatically matching the permission scope of the organization and all its subordinate organizations according to the organizational structure tree. For scheduled refresh tasks in the background, the permission scope is derived from the execution organization scope preset in the task configuration. For special calculation requests to a specific organization, the permission scope is derived from the set of organizations explicitly specified in the request parameters. This multi-source permission scope adaptation mechanism can cover the execution needs of all scenarios, including user interaction, background scheduling, and special calculations, ensuring that the corresponding data access boundaries can be accurately obtained under different triggering scenarios, avoiding data overreach or access restrictions caused by permission scope mismatch.

[0058] The obtained permission scope information is standardized and encoded, then converted into structured filtering conditions for data permission constraint fields. org The generation logic is as follows:

[0059] Wherein, `org_code` is the organization code value of the corresponding data permission constraint field in the data row, and `allowed_org_set` is the set of all organization codes allowed to access in the current operation context. This formula clarifies the permission filtering judgment logic through set matching, which can accurately define the scope of accessible data and avoid permission leakage or access omissions caused by fuzzy matching.

[0060] After generating the filtering conditions, they are injected into the pre-reserved data permission range injection position in the materialized refresh plan. This position corresponds to the first filtering node after the source table data is read in the query statement, ensuring that permission filtering takes effect immediately after data reading. By placing the filtering conditions at the initial stage of data processing, rather than filtering the results after calculation, data outside the permission range can be excluded from the data reading stage, significantly reducing the amount of data processing in subsequent calculation stages, improving overall execution efficiency, and preventing unauthorized data from entering the processing flow from the source of the calculation chain, thus ensuring the compliance of data access.

[0061] After the permission conditions are injected, the computation logic and storage operations of each node in the materialized refresh plan are executed sequentially according to the node execution order determined by the topology sorting. During execution, the data reading, intermediate calculation, and result output of each node are all performed under the constraints of the injected permission filtering conditions. All intermediate calculation results and final output results only contain data within the current permission range. After all nodes have been executed, the result table corresponding to the theme model is generated and persistently stored. The result table adopts a naming rule of logical table name and consistency identifier version number, corresponding one-to-one with the version on which consistency execution is based. At the same time, partition indexes and commonly used query dimension indexes are established for the billing period field and the organization field. The binding naming mechanism between the result and the configuration version can ensure that each result data can be traced back to the corresponding version of the configuration basis, meeting the auditability requirements under regulatory scenarios; the partition and index design can significantly improve the response speed of subsequent data queries and optimize the access performance of visualization display and detailed verification.

[0062] The above steps achieve precise control over data permissions throughout the materialized refresh process by dynamically acquiring the scope of permissions, pre-injecting filtering conditions, and constraining the entire execution process. This moves the traditional post-result filtering mode to the initial stage of data processing, effectively reducing the computational overhead of invalid data while ensuring data access compliance. At the same time, the binding and naming mechanism between results and configuration versions further enhances the traceability of the entire process, providing a compliant, accurate, and traceable result data foundation for subsequent dataset registration, visualization, and detailed verification.

[0063] For step S5, registering the result table as a queryable dataset, enabling the dashboard display stage to use the dataset as a data source for visual queries, includes: registering the result table as a queryable dataset, and configuring field metadata, organization scope field, payment period field, and a set of available query conditions for the dataset, so that the dashboard display stage can provide filtering controls to users based on the set of available query conditions, and dynamically generate query conditions based on the filtering values ​​selected by the user during visual queries; the organization scope field is used to adapt the data summarized in the unified data system according to regulatory standards to the internal entity organization level.

[0064] Furthermore, the visual query results of the dashboard display stage establish a traceable correspondence with the result table of the topic model through the consistency execution criteria.

[0065] Specifically, after the result table is materialized, the semantic configuration hash in the consistency execution basis is used as the version association identifier to complete the dataset registration. The registration information synchronously records the consistency identifier, physical table name, release version number, and other association information corresponding to the result table, establishing a one-to-one version mapping link between the visual query results and the underlying result table. This binding mechanism ensures that each visual display result can be traced back to the corresponding version's configuration basis and result data table, avoiding confusion of result data generated by different version configurations, and ensuring the auditability and traceability of the displayed data.

[0066] During registration, complete attribute information is configured for the dataset simultaneously: field metadata defines the name, data type, dictionary mapping, display format, and supported aggregation methods of each field, providing standardized field semantics for front-end display. Specifically, the "Organization Scope" field carries the mapping relationship between regulatory scope and internal entity hierarchy, used for subsequent permission filtering and organizational dimension screening, as well as adapting data summarized by regulatory scope in the unified data system to the internal entity hierarchy, supporting statistical analysis according to the internal management structure of the organization without repeatedly building subject models for different organizational levels. Simultaneously, the "Organization Scope" and "Accounting Period" fields mark the organizational and time dimension attributes of the dataset, respectively, for subsequent permission filtering and time range screening. Furthermore, query condition sets can define the supported filtering dimensions, operator types, and legal value ranges for the dataset, providing a configuration basis for the automatic generation of front-end filtering controls. Complete attribute configuration provides standardized data access semantics for the dashboard display, avoiding field ambiguity and permission risks caused by direct front-end manipulation of the underlying data tables.

[0067] After selecting this dataset as the data source in the dashboard presentation, the system automatically generates corresponding filter controls based on the available query criteria and displays them to the user. Users can freely choose dimensions, metrics, and filter values ​​to customize the displayed content. When a visualization query is triggered, the system parses the user-selected filter values, dimension groupings, aggregation methods, and sorting rules, dynamically generating corresponding structured query conditions. It then executes an aggregation query from the registered result table, returning the corresponding visualization results. This dynamically generated query approach flexibly responds to users' customized analysis needs, eliminating the need for pre-defined query logic and enhancing the flexibility and adaptability of the dashboard presentation.

[0068] The above steps establish a bridge between model calculation results and visualization by registering the materialized result table as a standardized queryable dataset. At the same time, the version binding mechanism based on consistency ensures the traceable correspondence between the displayed data and the calculation results, providing a unified data access entry and version benchmark for the business analysis dashboard and self-application configuration of the One-Table-Pass system application layer.

[0069] For step S6, when a detailed verification of the visualization query results is triggered, the display conditions at the summary result level are converted into detailed query conditions based on the data point context information of the chart where the triggering operation is located and the preset summary-detail field mapping rules. The query is then executed to return detailed data. The data point context information includes at least the chart type, click dimension field and corresponding value, hierarchical position, and the chart's own inherited filtering conditions. Following the priority order that temporary penetration configuration is superior to component penetration configuration, and component penetration configuration is superior to indicator definition condition tree, the source conditions in the data point context information are merged with the preset summary-detail field mapping rules to generate parameterized filtering conditions for the target detailed dataset as detailed query conditions.

[0070] Furthermore, when a detailed verification of the visualized query results is triggered, the version of the result table corresponding to the visualized query results and the version of the detailed dataset corresponding to the detailed data are located according to the consistency execution criteria.

[0071] Specifically, when a user clicks a chart data point on the dashboard interface to trigger detailed verification, the system first reads the consistency execution basis identifier associated with the current dashboard to locate the version of the topic model result table corresponding to the current visualization query result. Simultaneously, it matches the configuration version of the corresponding detailed dataset to ensure that the summary result and detailed data are generated based on the same configuration caliber. This version caliber identification mechanism avoids mismatches between the summary and detailed calibers caused by configuration version iterations, ensuring the consistency of the caliber and the accuracy of the results in the penetration verification.

[0072] After version identification is completed, the context information of the data points in the chart where the triggering operation is located is collected, and context elements such as chart type, click dimension fields and corresponding values, hierarchical position, and inherited filter conditions of the chart itself are fully extracted. The collection of multi-dimensional context information can completely retain the filter conditions and dimension attributes of the summary results, avoiding the loss of conditions caused by only passing click values, and providing a comprehensive contextual foundation for the subsequent complete reconstruction of detailed query conditions.

[0073] Subsequently, conditions are merged according to a preset priority rule, with the priority order being: temporary penetration configuration over component penetration configuration, and component penetration configuration over metric definition condition tree. Temporary penetration configuration, which is a filtering rule temporarily added by the user during the current penetration operation, has the highest priority, satisfying the customized needs of temporary verification. Component penetration configuration, which is a penetration rule pre-configured in the Kanban component, has the next highest priority, adapting to the fixed penetration logic at the component level. The metric definition condition tree, on the other hand, contains the filtering conditions defined by the metric itself, with the lowest priority, ensuring the consistency of the basic calculation criteria. This three-level priority rule reasonably balances customized needs with the consistency of basic criteria, avoiding conflicts between conditions from different sources.

[0074] During the condition merging process, based on preset summary-detail field mapping rules, the dimension fields and filter conditions on the summary side are mapped one by one to the corresponding fields in the detailed dataset. Operator mapping and value format conversion are completed simultaneously during the transformation, ultimately generating parameterized filter conditions for the target detailed dataset, which serve as the detailed query conditions. Standardized field mapping rules ensure that the conditions on the summary side are accurately converted into the filter logic on the detailed side, avoiding mismatches between detailed query results and summary data caused by field mismatches and condition deviations, thus guaranteeing the accuracy of the penetration verification.

[0075] After generating detailed query conditions, the detailed query is executed and the paginated detailed data is returned. For large result set queries with data volume exceeding the preset threshold, the system automatically switches to asynchronous export mode, generates result files, and notifies the user to download them, ensuring the stability and availability of large-scale detailed queries.

[0076] The above steps, through version positioning, multi-dimensional context collection, hierarchical priority condition merging, and standardized field mapping, achieve accurate conversion from summary display conditions to detailed query conditions. At the same time, relying on the version positioning mechanism based on consistency execution, the consistency of the summary and details is ensured. This not only supports the data drill-down verification needs of the application layer, but also provides detailed comparison capabilities for 1104 cross-system data quality verification. Meanwhile, it improves the accuracy and reliability of transparent details, and continues the traceability of the entire process, forming a complete closed loop from model construction and result display to detailed verification.

[0077] Overall, the proposed method for model building and detailed verification based on unified metadata in this invention connects the entire process of model building, verification and compilation, access control, visualization and detailed verification through unified metadata. It ensures consistent data standards and traceability throughout the entire process by establishing consistent execution criteria, thus adapting to the audit requirements of regulatory scenarios. By relying on pre-filtering of permissions, materialization and reuse of intermediate results, and topologically ordered execution, it effectively reduces unnecessary computational overhead and improves data processing efficiency. At the same time, it realizes a complete business closed loop from modeling, materialization, and display to verification, reducing the cost of implementing model results and possessing good scenario adaptability and engineering scalability.

[0078] Example 2: A model building and transparent detail system driven by metadata through a single table, such as... Figure 2 As shown, it includes: The OneTable metadata construction module 100 is used to receive the subject model processing configuration information and the transparent detailed verification configuration information based on the OneTable data system, and to convert the received configuration information into a OneTable metadata object with stable identifier, dependency relationship and data permission constraint fields. The subject model processing configuration information uses a directed acyclic graph to describe the data processing flow. The consistency verification and solidification module 200 is used to perform consistency verification on the one-table metadata object and solidify the verified metadata configuration to generate an immutable consistency execution basis. The materialized refresh plan compilation module 300 is used to compile the one-table metadata object into an executable materialized refresh plan based on the consistency execution basis, and to reserve injection positions for data permissions in the materialized refresh plan; The permission filtering and result materialization module 400 is used to filter the execution result of the materialized refresh plan based on the data permission constraint field, generate the topic model result table and save it. The dataset registration and visualization query module 500 is used to register the result table as a queryable dataset, so that the dashboard display can use the dataset as a data source for visualization query. The summary-detail penetration query module 600 is used to convert the display conditions at the summary result level into detailed query conditions based on the data point context information of the chart where the triggering operation is located and the preset summary-detail field mapping rules when the detailed verification of the visualization query results is triggered, and then execute the query to return detailed data.

[0079] The above description of the embodiments is provided to enable those skilled in the art to understand and apply the present invention. It will be apparent to those skilled in the art that various modifications can be made to the above embodiments, and the general principles described herein can be applied to other embodiments without inventive effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made to the present invention by those skilled in the art based on the disclosure thereof should be within the scope of protection of the present invention.

Claims

1. A method for model construction and detail penetration driven by metadata from a single table, characterized in that, Includes the following steps: S1. Receive the subject model processing configuration information and the transparent detailed verification configuration information based on the One Table Data System, and convert the received configuration information into a One Table metadata object with stable identifier, dependency relationship and data permission constraint fields. The subject model processing configuration information uses a directed acyclic graph to describe the data processing flow. S2. Perform consistency verification on the metadata object of the table, and solidify the metadata configuration that passes the verification to generate an immutable consistency execution basis; S3. Based on the consistency execution basis, compile the one-table metadata object into an executable materialized refresh plan, and reserve a data permission injection position in the materialized refresh plan; S4. Based on the data permission constraint field, perform permission filtering on the execution result of the materialized refresh plan, generate the theme model result table, and save it; S5. Register the result table as a queryable dataset so that the dashboard display can use the dataset as a data source for visual querying; S6. When a detailed verification of the visual query results is triggered, the display conditions at the summary result level are converted into detailed query conditions based on the data point context information of the chart where the triggering operation is located and the preset summary-detail field mapping rules, and the query is executed to return detailed data.

2. The method for model construction and detail penetration based on metadata driven by a single table, as described in claim 1, is characterized in that... The unified data system includes at least a resource layer, a middle layer detailed dimension table, and a middle layer summary dimension table. The resource layer provides standardized regulatory detailed data. The middle layer detailed dimension table reconstructs the standardized regulatory detailed data of the resource layer according to the regulatory business theme to form detailed dimension data. The middle layer summary dimension table summarizes the middle layer detailed dimension table to form summary dimension data. The dependencies include upstream and downstream dependencies between processing nodes in the topic model, input field source relationships, and mapping relationships between summary dataset fields and detail dataset fields in the transparent detail dataset. The upstream and downstream dependencies include at least the dependencies of adapting the intermediate layer summary dimension table based on the intermediate layer detail dimension table generation mechanism.

3. The method for model construction and detail penetration based on metadata driven by a single table, as described in claim 1, is characterized in that... The process of performing consistency verification on the metadata object of the table, and solidifying the verified metadata configuration to generate an immutable consistency execution basis, includes the following steps: Perform node uniqueness verification, endpoint validity verification, acyclicity verification, unique result node verification, node field reference validity verification, and data permission field legality verification on the directed acyclic graph in the aforementioned metadata object; After verification, the semantic configuration that affects the data calculation results is separated from the layout configuration that only affects the display effect. Then, normalization and hash calculation are performed. The hash value of the semantic configuration is used as a consistency identifier. The consistency identifier is associated and bound with the verification report, approval record, release version and running task to generate an immutable consistency execution basis.

4. The method for model construction and detail penetration based on metadata driven by a single table, as described in claim 1, is characterized in that... The step of compiling the table-wide metadata object into an executable materialized refresh plan based on the consistency execution criteria includes the following steps: Based on the fixed configuration version identified by the consistency execution basis, the structured query language fragments corresponding to each processing node are recursively generated according to the dependency order of each processing node in the one-table metadata object. Based on the type of each processing node, corresponding query clauses or data definition statements are generated to form the calculation logic of each processing node; Based on the computational logic, storage operation statements are generated for the nodes marked as needing materialization in each of the processing nodes. The storage operation statements include table creation statements, time-period deletion statements, data insertion statements, and index creation statements. The materialization refresh plan includes the computational logic of each processing node and the storage operation statements of the nodes marked as needing materialization.

5. The method for model construction and detail penetration based on metadata driven by a single table, as described in claim 1, is characterized in that... The step of filtering the execution results of the materialized refresh plan based on the data permission constraint fields, generating the theme model result table, and saving it includes the following steps: Retrieve permission scope information in the current operation context; The permission scope information is converted into filtering conditions for the data permission constraint field, and the filtering conditions are injected into the reserved data permission scope injection position in the materialized refresh plan, so that the execution process of the materialized refresh plan only processes the data that is allowed to be accessed by the current operation context; Execute the materialization refresh plan, generate the result table corresponding to the theme model, and save it.

6. The method for model construction and detail penetration based on metadata driven by a single table, as described in claim 1, is characterized in that... Registering the result table as a queryable dataset includes the following steps: Register the result table as a queryable dataset, and configure field metadata, organization scope field, payment period field and available query condition set for the dataset, so that the dashboard display can provide users with filtering controls based on the available query condition set, and dynamically generate query conditions based on the filter values ​​selected by the user during the visual query. The "Organization Scope" field is used to adapt the data summarized in the "One Table Data System" according to regulatory standards to the internal entity level of the organization.

7. The method for model construction and detail penetration based on metadata driven by a single table, as described in claim 1, is characterized in that... The process of converting the display conditions at the summary result level into detailed query conditions based on the data point context information of the chart where the triggering operation is located and the preset summary-detail field mapping rules includes the following steps: The data point context information includes at least the chart type, click dimension field and corresponding value, hierarchical position, and the chart's own inherited filtering conditions. Following the priority order that temporary penetration configuration is superior to component penetration configuration, and component penetration configuration is superior to indicator definition condition tree, the source conditions in the data point context information are merged with the preset summary-detail field mapping rules to generate parameterized filtering conditions for the target detail dataset as detail query conditions.

8. The method for model construction and detail penetration based on metadata driven by a single table, as described in claim 1, is characterized in that... Before compiling and generating the materialized refresh plan, the following is also included: The processing nodes are sorted according to the topological order of the processing nodes in the directed acyclic graph. When test run mode is enabled, in response to the instruction to start running from a specified node, only the specified node and its downstream nodes are run, and the associated downstream nodes that are not run are marked as expired.

9. The method for model construction and detail penetration based on metadata driven by a single table, as described in claim 1, is characterized in that... The visual query results of the dashboard display are linked to the result table of the theme model through the consistency execution criteria to establish a traceable correspondence. When a detailed verification of the visualized query results is triggered, the version of the result table corresponding to the visualized query results and the version of the detailed dataset corresponding to the detailed data are located according to the consistency execution criteria.

10. A model construction and transparent detail system based on metadata driven by a single table, characterized in that, The system is used to implement the model construction and detail penetration method based on single-table metadata as described in claims 1-9, the system comprising: The OneTable metadata construction module is used to receive the subject model processing configuration information and the transparent detailed verification configuration information based on the OneTable data system. It transforms the received configuration information into a OneTable metadata object with stable identifier, dependency relationship and data permission constraint fields. The subject model processing configuration information uses a directed acyclic graph to describe the data processing flow. The consistency verification and solidification module is used to perform consistency verification on the metadata object of the table and solidify the metadata configuration that passes the verification to generate an immutable consistency execution basis. The materialized refresh plan compilation module is used to compile the one-table-through metadata object into an executable materialized refresh plan based on the consistency execution basis, and to reserve injection positions for data permissions in the materialized refresh plan; The permission filtering and result materialization module is used to filter the execution results of the materialized refresh plan based on the data permission constraint field, generate the topic model result table and save it. The dataset registration and visualization query module is used to register the result table as a queryable dataset, so that the dashboard display can use the dataset as a data source for visualization query. The summary-detail penetration query module is used to convert the display conditions at the summary result level into detailed query conditions when the detailed verification of the visual query results is triggered, based on the data point context information of the chart where the triggering operation is located and the preset summary-detail field mapping rules, and then execute the query to return detailed data.

Images