A data production and application method based on index management

Abstract

The application discloses a kind of data production and application method based on index management, by receiving the index query request including target index identification, dimension constraint condition and time range parameter, based on global index asset library analysis and semantic verification target index identification, obtain index definition information;According to index definition information, generate standardized query statement, and select optimal data calculation engine to execute query;To the query process implements multilevel cache optimization and parallel computing acceleration, obtains original data set;Original data set is in real time processing according to preset analysis model, generates the structured index data containing trend analysis, anomaly detection or attribution inference result;Based on fine-grained permission control strategy filtering and desensitization data, only return the content in permission range and record audit log.The method realizes the unified management of index data, efficient query and intelligent analysis, improves the automation level and query performance of data production, while guaranteeing data security and access controllability.

Description

Technical Field

[0001] This invention relates to the field of data management technology, and in particular to a data production and application method based on indicator management. Background Technology

[0002] In the field of modern enterprise data management and analysis, metrics serve as the core basis for quantifying business operations and performance, and their efficient and reliable data supply is crucial for business decision-making. Currently, the traditional approach commonly used in the industry relies on a combination of data warehouses and business intelligence tools. Technical teams manually write query scripts based on business needs, extracting data from underlying data tables and generating static reports or exporting files for use. While this approach supported basic data analysis needs for a considerable period, the lack of a unified business semantic layer and automated production mechanisms resulted in long metric development cycles, chaotic definition management, and difficulty in adapting to the ever-increasing demands for multi-dimensional, real-time data analysis. Especially when facing large-scale data volumes and complex business logic, query performance often becomes a bottleneck, failing to support high-concurrency and low-latency interactive analysis scenarios. Furthermore, during data sharing and use, coarse-grained access control and inadequate auditing mechanisms also increase data security and management complexity. Summary of the Invention

[0003] In view of this, the purpose of this invention is to propose a data production and application method based on indicator management. By constructing a global indicator asset library, semantic unity and automated query generation are achieved. Combined with intelligent routing and caching acceleration mechanisms, this method solves the problems of inefficient enterprise indicator data production, inconsistent definitions, and insufficient query performance.

[0004] To achieve the above-mentioned technical objectives, the technical solution adopted by this invention is: a data production and application method based on indicator management, comprising:

[0005] Receive indicator query requests, which include the target indicator identifier, dimension constraints, and time range parameters;

[0006] Based on the global indicator asset library, the target indicator identifier is parsed and semantically verified to obtain the verified indicator definition information. The global indicator asset library stores the definition information of multiple global indicators. Each global indicator includes atomic indicators, derived indicators, and composite indicators. The definition information includes the business scope, technical calculation logic, associated data source, and dimensions of the indicator.

[0007] Based on the indicator definition information, a standardized query statement adapted to the underlying data warehouse structure is generated through the indicator-data mapping relationship, and the optimal data computing engine is selected to execute the query according to the intelligent routing strategy;

[0008] Multi-level caching optimization and parallel computing acceleration are applied to the query execution process to obtain the original dataset. Multi-level caching optimization includes pre-warming and dynamic updating of query results for high-frequency indicators. Parallel computing acceleration adopts MPP architecture for distributed processing.

[0009] The original dataset is processed in real time according to the preset analysis model to generate structured indicator data with trend analysis, anomaly detection or attribution inference results;

[0010] Based on fine-grained access control strategies, structured indicator data is filtered and anonymized, returning only the data content accessible to the requester within their authorized scope, and recording the complete data access behavior in the audit log for traceability analysis.

[0011] In some embodiments, the global metrics asset library is generated through the following steps:

[0012] Receive indicator registration requests. The indicator registration request includes the business definition, technical calculation logic, associated data source and dimension information of the indicator to be registered.

[0013] Based on the reuse level of the technical computing logic, the indicators to be registered are classified into atomic indicators, derived indicators, or composite indicators.

[0014] If the metric to be registered is an atomic metric, it is mapped and modeled to the measurement fields and their default aggregation methods in the core fact table of the data warehouse;

[0015] If the indicator to be registered is a derived indicator or a composite indicator, then based on the defined atomic indicator or derived indicator, a derivative model is performed by configuring dimensions, filtering conditions and calculation formulas.

[0016] The mapping relationship between the indicators to be registered output after the above modeling and the data warehouse tables, fields and calculation rules is persisted to form a traceable indicator-data lineage and form indicator definition information.

[0017] Store indicator definition information in a global indicator asset library and assign a globally unique version identifier to the indicators associated with the indicator definition information.

[0018] Perform lifecycle management on indicator definition information in the global indicator asset library, including automated inspection and archiving based on access frequency and business relevance.

[0019] In some embodiments, the target indicator identifier is parsed and semantically validated based on a global indicator asset library to obtain validated indicator definition information, including:

[0020] Retrieve complete indicator definition information associated with the identifier from the global indicator asset library;

[0021] Perform a grammatical compliance check on the retrieved indicator definition information to verify the completeness and standardization of the business scope, technical calculation logic, associated data sources, and dimensional information of the retrieved indicator definition information;

[0022] Based on the preset semantic rule base for indicators, the semantic consistency of the indicator definition information that has passed the syntax check is verified to ensure that there is no contradiction or conflict between its business definition and technical calculation logic.

[0023] When semantic ambiguity or conflict is detected in the indicator definition information, the indicator caliber anomaly handling process is triggered to generate a verification report containing specific conflict items and correction suggestions.

[0024] A digital signature and timestamp are added to the indicator definition information that passes all verifications to generate verified indicator definition information that can be trusted by subsequent query and execution components.

[0025] In some embodiments, a standardized query statement adapted to the underlying data warehouse structure is generated based on the indicator definition information and the indicator-data mapping relationship, including:

[0026] Parse the validated indicator definition information and extract the technical calculation logic, the identifiers of the associated data sources, and the dimension information as query generation elements;

[0027] Based on the query-generated elements, the corresponding data warehouse tables, field names, and relationships are determined through indicator-data mapping relationships;

[0028] Based on the business scope and technical calculation logic of the indicator definition information, the dimensional constraints and time range parameters are converted into corresponding SQL filter clauses.

[0029] Based on the optimization principles of the data warehouse hierarchical model, the optimal data level is automatically selected and a complete query body containing a select list, group by grouping, and aggregate functions is constructed.

[0030] The generated query statements are optimized using an abstract syntax tree-based query optimization algorithm. This eliminates redundant calculations and ensures that the syntax conforms to the target computing engine specifications, outputting optimized and standardized query statements that are fully adapted to the underlying data warehouse structure.

[0031] In some embodiments, selecting the optimal data computing engine to execute the query based on an intelligent routing strategy includes:

[0032] Obtain the query characteristics of standardized query statements, including syntax features, data volume estimation results, and complexity scores;

[0033] Based on query statement feature matching, a pre-configured engine capability profile is formed, which includes the performance and resource consumption characteristics of each data computing engine for a specific query type.

[0034] By combining real-time load monitoring data, a multi-objective optimization algorithm is used to comprehensively evaluate query latency, resource utilization, and computing cost to obtain the evaluation results.

[0035] Based on the evaluation results, the optimal target execution engine is selected from the available engine cluster, and corresponding query task distribution instructions are generated;

[0036] Establish a session connection between the query task and the execution engine to complete the initial configuration of the query context environment.

[0037] In some embodiments, multi-level caching optimization and parallel computing acceleration are applied to the query execution process to obtain the original dataset, including:

[0038] Before the query is executed, a cache matching search is performed based on the characteristics of the query statement and the historical execution records. If the result cache is hit, the cached data is returned directly.

[0039] For query requests that miss the cache, a cache preheating strategy based on a cost model is adopted to identify intermediate results corresponding to high-frequency access patterns and pre-compute and store them.

[0040] Complex query tasks are broken down into multiple sub-query units that can be executed in parallel, and the optimal parallel execution plan is generated by the query optimizer.

[0041] A dynamic resource scheduling algorithm is used to allocate appropriate computing resources to each sub-query unit, intelligent caching management is implemented for intermediate results during execution, and a mapping relationship between query statement features and cached data is established.

[0042] The final query results are compressed and stored, and the cached metadata information is updated to generate the original dataset.

[0043] In some embodiments, a dynamic resource scheduling algorithm is used to allocate appropriate computing resources to each sub-query unit, intelligent caching management is implemented for intermediate results during execution, and a mapping relationship between query statement features and cached data is established, including:

[0044] Based on the computational complexity characteristics of subquery units and the data dependency graph, a multi-objective optimization algorithm is used to balance resource allocation among computing nodes.

[0045] The resource allocation strategy is dynamically adjusted based on the actual execution progress of each unit.

[0046] The intermediate result data generated during the execution process is valued, and the cache priority is determined based on the access popularity prediction model.

[0047] Construct a mapping index between query feature vectors and cached data blocks, and use a consistent hashing algorithm to achieve efficient location of distributed cache;

[0048] In addition, cached content is dynamically maintained based on data access frequency and freshness requirements.

[0049] In some embodiments, the original dataset is processed in real time according to a preset analysis model to generate structured indicator data with trend analysis, anomaly detection, or attribution inference results, including:

[0050] Perform data quality checks and missing value handling on the original dataset to ensure the integrity and accuracy of the input data;

[0051] Select the corresponding analysis model based on the business scope of the indicator definition information, including time series forecasting model, isolated forest anomaly detection model or SHAP attribution analysis model;

[0052] The preprocessed raw dataset is input into the selected analysis model to perform batch or streaming computation and generate preliminary analysis results.

[0053] The preliminary analysis results are post-processed and optimized, including result smoothing, confidence interval calculation, and significance labeling.

[0054] The optimized preliminary analysis results are linked and integrated with the original indicator data to generate structured indicator data that includes basic data and analysis conclusions.

[0055] In some embodiments, filtering and de-identifying structured indicator data based on fine-grained access control strategies includes:

[0056] Parse the requester's identity credentials and access context, and extract permission attributes, including the user identifier, role permissions, and data access scope constraints corresponding to the requester.

[0057] Based on the extracted permission attributes, match them with a predefined set of access control rules to determine the scope of access the requester can have to the target metric data.

[0058] Based on the defined access scope, row and column level data filtering is performed on structured indicator data to remove data content outside the requester's permission scope;

[0059] Apply dynamic desensitization algorithms to sensitive fields in the retained data, and implement differentiated desensitization processing based on the field sensitivity level and user permissions;

[0060] Generate audit log records containing details of data filtering and de-identification, establish correlations with the processed indicator data, and construct standardized data response packages.

[0061] In some embodiments, the system returns the data content accessible within the requester's permissions and records the complete data access behavior to the audit log for tracing and analysis, including:

[0062] The standardized data response packet is returned to the requester through a secure transmission channel, and data fingerprint verification information is added to the response header of the standardized data response packet.

[0063] Capture the entire data access process in real time, including the requester's identity, access time, query conditions, returned data volume, and processing time.

[0064] The captured end-to-end behavioral information is associated and integrated with the corresponding indicator definition information and access control policies to generate structured audit log records.

[0065] The audit logs are encrypted and stored using blockchain technology.

[0066] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art: It receives indicator query requests containing target indicator identifiers, dimensional constraints, and time range parameters; based on the global indicator asset library, it parses and semantically verifies the target indicator identifiers to obtain indicator definition information; it generates standardized query statements based on the indicator definition information and selects the optimal data computing engine to execute the query; it implements multi-level caching optimization and parallel computing acceleration for the query process to obtain the original dataset; it processes the original dataset in real time according to a preset analysis model to generate structured indicator data containing trend analysis, anomaly detection, or attribution inference results; and it filters and de-identifies data based on fine-grained access control strategies, returning only content within the authorized scope and recording audit logs. This method achieves unified management, efficient querying, and intelligent analysis of indicator data, improves the automation level of data production and query performance, and simultaneously ensures data security and access controllability. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 This is a flowchart illustrating steps S101 to S106 of the method described in the specific implementation embodiment;

[0069] Figure 2 This is a flowchart illustrating steps S201 to S206 of the method described in the specific implementation. Detailed Implementation

[0070] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be particularly noted that the following embodiments are for illustrative purposes only and do not limit the scope of the invention. Similarly, the following embodiments are only some, not all, embodiments of the present invention, and all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] Please see Figure 1 This embodiment provides a data production and application method based on indicator management, including:

[0072] S101. Receive indicator query request. The indicator query request includes the target indicator identifier, dimension constraints and time range parameters.

[0073] S102. Based on the global indicator asset library, the target indicator identifier is parsed and semantically verified to obtain the verified indicator definition information. The global indicator asset library stores the definition information of multiple global indicators. Each global indicator includes atomic indicators, derived indicators and composite indicators. The definition information includes the business scope, technical calculation logic, associated data source and dimension of the indicator.

[0074] S103. Based on the indicator definition information, generate standardized query statements that are adapted to the underlying data warehouse structure through the indicator-data mapping relationship, and select the optimal data computing engine to execute the query according to the intelligent routing strategy.

[0075] S104. Apply multi-level caching optimization and parallel computing acceleration to the query execution process to obtain the original dataset. Multi-level caching optimization includes pre-warming storage and dynamic updating of query results for high-frequency indicators. Parallel computing acceleration adopts MPP architecture for distributed processing.

[0076] S105. Process the original dataset in real time according to the preset analysis model to generate structured index data with trend analysis, anomaly detection or attribution inference results;

[0077] S106. Based on fine-grained access control strategies, structured indicator data is filtered and anonymized, and only the data content accessible within the requester's permission scope is returned. The complete data access behavior is recorded in the audit log for traceability analysis.

[0078] In step S101, the indicator query request is a standardized data retrieval instruction initiated by the business user. The target indicator identifier is used to uniquely identify the indicator object to be queried, the dimension constraint is used to limit the grouping and filtering range of the data, and the time range parameter is used to specify the time interval of the data. This step receives the request through a unified interface, providing input for subsequent automated processing.

[0079] In step S102, the global indicator asset library is the core component that centrally stores and manages all indicator definitions. Atomic indicators are indivisible basic metrics. Derived indicators are derived from atomic indicators through dimensional constraints or simple calculations. Composite indicators are obtained by combining multiple indicators. The business definition describes the business meaning and rules of the indicators, the technical calculation logic defines the specific calculation method of the indicators, and the associated data source and dimensions indicate the data tables and dimension attributes that the indicators depend on. The parsing and semantic verification process ensures the completeness and consistency of the indicator definitions through verification algorithms, avoiding ambiguity in the definitions.

[0080] In step S103, the indicator-data mapping relationship establishes the association between indicator definitions and underlying data warehouse tables, fields, and calculation rules, which is used to automatically generate standardized query statements. The intelligent routing strategy dynamically selects the optimal data computing engine (such as Presto or Spark) based on query complexity, data volume, and engine load status to improve query efficiency and resource utilization.

[0081] In step S104, multi-level caching optimization identifies high-frequency query patterns and performs pre-warming storage and dynamic updates of query results, reducing redundant computation. Parallel computing acceleration uses an MPP architecture to decompose the query task into multiple subtasks for parallel execution, significantly improving the processing speed of large-scale data queries.

[0082] In step S105, the preset analysis model includes standardized analysis algorithms such as trend analysis, anomaly detection, and attribution inference, which are used to process the original dataset in real time, generate structured indicator data with analysis conclusions, and enhance the business insight of the data.

[0083] In step S106, the fine-grained access control policy, based on user roles and data sensitivity levels, implements row- and column-level filtering and dynamic anonymization of structured indicator data to ensure data security. Audit logs record complete data access behavior, supporting operation tracing and compliance auditing.

[0084] This embodiment receives indicator query requests, parses and verifies them based on a global indicator asset database, automatically generates standardized query statements, selects the optimal computing engine for execution, and combines caching optimization and parallel computing to accelerate data acquisition. After real-time analysis and processing, the results are returned according to access control. This method achieves unified indicator definitions, efficient and automated querying, and controllable data security through end-to-end automated processing, effectively improving data production efficiency and quality.

[0085] Please see Figure 2 In some embodiments, the global metrics asset library is generated through the following steps:

[0086] S201. Receive indicator registration request. The indicator registration request includes the business definition, technical calculation logic, associated data source and dimension information of the indicator to be registered.

[0087] S202. Based on the reuse level of the technical computing logic, the indicators to be registered are classified into atomic indicators, derived indicators, or composite indicators.

[0088] S203. If the indicator to be registered is an atomic indicator, model its mapping with the measurement fields and their default aggregation methods in the core fact table of the data warehouse.

[0089] If the indicator to be registered is a derived indicator or a composite indicator, then based on the defined atomic indicator or derived indicator, a derivative model is performed by configuring dimensions, filtering conditions and calculation formulas.

[0090] S204. Persist the mapping relationship between the indicators to be registered output after the above modeling and the data warehouse tables, fields and calculation rules to form a traceable indicator-data lineage association and form indicator definition information.

[0091] S205. Store the indicator definition information in the global indicator asset library and assign a globally unique version identifier to the indicator associated with the indicator definition information.

[0092] S206. Perform lifecycle management on indicator definition information in the global indicator asset library, including automated inspection and archiving based on access frequency and business relevance.

[0093] In step S201, the indicator registration request is a standardized indicator creation application submitted by the indicator definer. It includes core metadata such as business definition, technical calculation logic, associated data sources, and dimensional information. The business definition describes the business meaning and statistical rules of the indicator; the technical calculation logic defines the specific calculation expression and data processing flow of the indicator; and the associated data sources and dimensional information specify the data tables and dimensional attributes on which the indicator depends. This step ensures the standardization and completeness of the indicator definition.

[0094] In step S202, the reuse level is determined based on the dependencies and complexity of the technical calculation logic: atomic metrics are indivisible basic measures that directly originate from the measurement fields of the core fact table in the data warehouse; derived metrics are derived from atomic metrics through dimensional constraints, filtering conditions, or simple calculations; composite metrics are obtained by combining multiple atomic or derived metrics. The classification process is completed automatically by the rule engine, ensuring a clear hierarchy and reusability of the indicator system.

[0095] In step S203, mapping modeling establishes a direct association between atomic metrics and the core fact table's measurement fields and their default aggregation methods (such as SUM and COUNT); derivation modeling, for derived and composite metrics, flexibly combines metric logic based on defined atomic or derived metrics by configuring dimensions, filtering conditions, and calculation formulas. Preferably, a visual configuration interface is provided to support business personnel in defining metrics, reducing the technical barrier.

[0096] In step S204, the mapping relationship persistence stores the association information between the metric and the data warehouse tables, fields, and calculation rules in the metadata management system, forming a traceable metric-data lineage. This lineage allows for tracing the data source of the metric upstream and tracking the impact of data usage downstream, enhancing data transparency and credibility.

[0097] In step S205, a globally unique version identifier is assigned through an identifier generation algorithm to ensure the uniqueness of the indicator definition information and version traceability. The version identifier supports iterative updates of indicator definitions and historical version management, avoiding data inconsistencies caused by indicator changes.

[0098] In step S206, lifecycle management sets automated inspection rules based on dimensions such as access frequency and business relevance, identifies low-usage indicators, and triggers the archiving process to ensure the simplicity and efficiency of the indicator asset library. Automated inspection is executed through scheduled tasks, reducing manual maintenance costs.

[0099] This embodiment constructs a unified and traceable global indicator asset library through standardized indicator registration, classification modeling, and lineage management, providing basic support for the consistent management and efficient reuse of indicators, and effectively solving the problems of inconsistent indicator definitions and redundant development.

[0100] In some embodiments, the target indicator identifier is parsed and semantically validated based on a global indicator asset library to obtain validated indicator definition information, including:

[0101] Retrieve complete indicator definition information associated with the identifier from the global indicator asset library;

[0102] Perform a grammatical compliance check on the retrieved indicator definition information to verify the completeness and standardization of the business scope, technical calculation logic, associated data sources, and dimensional information of the retrieved indicator definition information;

[0103] Based on the preset semantic rule base for indicators, the semantic consistency of the indicator definition information that has passed the syntax check is verified to ensure that there is no contradiction or conflict between its business definition and technical calculation logic.

[0104] When semantic ambiguity or conflict is detected in the indicator definition information, the indicator caliber anomaly handling process is triggered to generate a verification report containing specific conflict items and correction suggestions.

[0105] A digital signature and timestamp are added to the indicator definition information that passes all verifications to generate verified indicator definition information that can be trusted by subsequent query and execution components.

[0106] In this embodiment, retrieving the complete indicator definition information associated with the identifier from the global indicator asset library can be understood as performing a precise matching query in the asset library through the indicator identifier to obtain complete metadata including business scope, technical calculation logic, associated data sources and dimension information. This step ensures that subsequent processing is based on accurate indicator definitions.

[0107] The syntax compliance check verifies the completeness and standardization of the indicator definition information; the business definition check ensures that the business description is unambiguous and conforms to naming conventions; the technical calculation logic verification ensures that the calculation expression is syntactically correct and executable; and the related data source and dimension information check confirms that the data table structure and dimension attributes exist and are accessible. This check is automatically executed through a predefined set of syntax rules to prevent calculation errors caused by incomplete definitions.

[0108] The semantic rule base stores the mapping relationship between business rules and technical rules. Semantic consistency verification compares the degree of matching between business descriptions and technical calculation logic through the rule engine to ensure that there are no contradictions or conflicts. For example, it verifies whether the business definition of "order amount" is consistent with the SUM aggregation logic in the technical calculation, avoiding semantic ambiguity.

[0109] When semantic ambiguity or conflict is detected, the indicator caliber anomaly handling process is automatically triggered. A pattern recognition algorithm locates the specific conflicting item, and a verification report containing correction suggestions is generated based on a rule base. Preferably, this process supports manual review to ensure the accuracy of anomaly handling.

[0110] Digital signatures and timestamps attach a trusted identifier to verified metric definition information using asymmetric encryption algorithms and time services. Digital signatures ensure information integrity and provenance, while timestamps record the verification time. This allows subsequent query execution components to directly trust and use the metric definition information without requiring repeated verification.

[0111] This embodiment ensures the consistency of indicator definitions at the syntactic and semantic levels through a multi-layered verification mechanism, and establishes a trusted transmission mechanism by combining digital signature technology, fundamentally guaranteeing the accuracy and reliability of indicator data and effectively solving the problem of inconsistent standards across departments.

[0112] In some embodiments, a standardized query statement adapted to the underlying data warehouse structure is generated based on the indicator definition information and the indicator-data mapping relationship, including:

[0113] Parse the validated indicator definition information and extract the technical calculation logic, the identifiers of the associated data sources, and the dimension information as query generation elements;

[0114] Based on the query-generated elements, the corresponding data warehouse tables, field names, and relationships are determined through indicator-data mapping relationships;

[0115] Based on the business scope and technical calculation logic of the indicator definition information, the dimensional constraints and time range parameters are converted into corresponding SQL filter clauses.

[0116] Based on the optimization principles of the data warehouse hierarchical model, the optimal data level is automatically selected and a complete query body containing a select list, group by grouping, and aggregate functions is constructed.

[0117] The generated query statements are optimized using an abstract syntax tree-based query optimization algorithm. This eliminates redundant calculations and ensures that the syntax conforms to the target computing engine specifications, outputting optimized and standardized query statements that are fully adapted to the underlying data warehouse structure.

[0118] In this embodiment, the semantically validated indicator definition is parsed in a structured manner, and the technical calculation logic, the identifier of the associated data source, and the dimension information are extracted as query generation elements. The technical calculation logic includes the calculation expression and aggregation method of the indicator, the identifier of the associated data source indicates the physical table where the data is located, and the dimension information determines the grouping and filtering dimensions of the data.

[0119] Determining the corresponding data warehouse table, field names, and relationships based on query-generated elements through indicator-data mapping refers to converting logical indicator elements into physical data objects using predefined mapping rules. This mapping relationship maintains the correspondence between indicators and underlying data table fields, ensuring that queries can correctly associate with the actual data storage location.

[0120] The dimensional constraints and time range parameters are converted into corresponding SQL filter clauses. A condition parser then transforms the business-level filtering conditions into executable WHERE clauses. Dimensional constraints are converted into equality or range filters, and time range parameters are converted into interval conditions for time fields, ensuring that the query results meet the business filtering requirements.

[0121] The system automatically selects the optimal data layer based on the data warehouse layering model optimization principles, intelligently choosing to retrieve data from the DWD detail layer, DWS summary layer, or ADS application layer according to query complexity, data volume, and performance requirements. This selection is based on a comprehensive evaluation of data freshness, query performance, and data volume, constructing a complete query body that includes select lists, group by grouping, and aggregate functions.

[0122] A query optimization algorithm based on abstract syntax trees is used to optimize the structure of the generated query statements. By parsing the syntax tree structure of the SQL statements, redundant calculations are eliminated, join order is optimized, and predicate pushdown is performed to ensure that the syntax conforms to the target computing engine specifications. This optimization process outputs standardized query statements that are fully adapted to the underlying data warehouse structure, thereby improving query execution efficiency.

[0123] This embodiment uses an automated query statement generation and optimization mechanism to convert business indicator requirements into efficient standard SQL queries, achieving a seamless transition from business logic to data querying and effectively solving the problems of low efficiency and error-proneness of manually writing SQL.

[0124] In some embodiments, selecting the optimal data computing engine to execute the query based on an intelligent routing strategy includes:

[0125] Obtain the query characteristics of standardized query statements, including syntax features, data volume estimation results, and complexity scores;

[0126] Based on query statement feature matching, a pre-configured engine capability profile is formed, which includes the performance and resource consumption characteristics of each data computing engine for a specific query type.

[0127] By combining real-time load monitoring data, a multi-objective optimization algorithm is used to comprehensively evaluate query latency, resource utilization, and computing cost to obtain the evaluation results.

[0128] Based on the evaluation results, the optimal target execution engine is selected from the available engine cluster, and corresponding query task distribution instructions are generated;

[0129] Establish a session connection between the query task and the execution engine to complete the initial configuration of the query context environment.

[0130] In this embodiment, query statement features refer to the set of features obtained by parsing and statistically analyzing standardized query statements. The syntactic features reflect the structural characteristics of the query statement, the data volume estimation result estimates the data scale involved in the query based on statistical information, and the complexity score comprehensively evaluates the computational complexity of the query.

[0131] An engine capability profile is a description of the capabilities of each data computing engine, built through the collection and analysis of historical performance data. It includes performance characteristics and resource consumption features for specific query types. Performance metrics cover indicators such as query response time and throughput, while resource consumption features include usage patterns for resources such as CPU, memory, and I / O. Preferably, the engine capability profile is continuously updated and optimized from historical execution records using machine learning methods.

[0132] Multi-objective optimization algorithms are mathematical optimization methods that weigh multiple objectives such as query latency, resource utilization, and computational cost. By establishing an objective function and constraints, and combining real-time load monitoring data, a comprehensive evaluation result is calculated. This algorithm employs optimization strategies such as weighted summation or Pareto optimality to ensure that the selected result achieves a balance across multiple performance dimensions.

[0133] The engine scheduling instructions generated based on the evaluation results include information such as the target execution engine identifier, query statement, and execution parameters. The query task distribution instructions are sent to the target engine via message queues or API calls to ensure that query tasks are accurately distributed to the optimal execution node.

[0134] A session connection is a communication link established between a query task and the execution engine, used to transmit query requests and return results. The initial configuration of the query context includes setting connection parameters, loading necessary data dictionaries, and pre-allocating computing resources to ensure the query can be executed in an optimal environment.

[0135] This embodiment achieves optimal matching between query tasks and computing engines through an intelligent engine selection mechanism, giving full play to the strengths of different engines, optimizing resource utilization while ensuring query performance, and effectively improving the overall system's query efficiency and resource utilization.

[0136] In some embodiments, multi-level caching optimization and parallel computing acceleration are applied to the query execution process to obtain the original dataset, including:

[0137] Before the query is executed, a cache matching search is performed based on the characteristics of the query statement and the historical execution records. If the result cache is hit, the cached data is returned directly.

[0138] For query requests that miss the cache, a cache preheating strategy based on a cost model is adopted to identify intermediate results corresponding to high-frequency access patterns and pre-compute and store them.

[0139] Complex query tasks are broken down into multiple sub-query units that can be executed in parallel, and the optimal parallel execution plan is generated by the query optimizer.

[0140] A dynamic resource scheduling algorithm is used to allocate appropriate computing resources to each sub-query unit, intelligent caching management is implemented for intermediate results during execution, and a mapping relationship between query statement features and cached data is established.

[0141] The final query results are compressed and stored, and the cached metadata information is updated to generate the original dataset.

[0142] In this embodiment, cache matching retrieval is a process of determining whether existing cached data can be reused by comparing and analyzing query statement features with historical execution records. Query statement features include elements such as syntax structure, filtering conditions, and aggregation patterns, while historical execution records store cache hit information from past queries. This retrieval achieves fast matching through feature similarity calculation; if a cached result is hit, the cached data is directly returned, avoiding redundant calculations.

[0143] The cost-based cache preheating strategy refers to intelligently predicting and pre-calculating intermediate results corresponding to high-frequency access patterns based on factors such as query frequency, data update cycle, and computational cost. This strategy identifies hot data by analyzing historical query patterns and pre-calculates and stores it during periods of low system load, effectively improving cache hit rate and query response speed.

[0144] The query optimizer determines the optimal parallelism based on data distribution characteristics and computing resource availability through cost evaluation, ensuring that each sub-query unit can execute efficiently and collaboratively. The dynamic resource scheduling algorithm allocates appropriate computing resources in real time according to the computational complexity and priority of each sub-query unit. This algorithm monitors system resource utilization, intelligently adjusts the allocation of resources such as CPU and memory, and implements intelligent caching management for intermediate results during execution, establishing a mapping relationship between query statement characteristics and cached data to achieve elastic allocation and efficient utilization of resources.

[0145] The final query results are compressed using algorithms such as columnar compression or dictionary encoding to reduce storage space. Simultaneously, cached metadata information is updated to record attributes such as data storage location, compression method, and expiration date. This process generates a structured raw dataset, providing an efficient storage foundation for subsequent data applications.

[0146] This embodiment achieves a significant improvement in query performance by organically combining multi-level caching optimization with parallel computing acceleration. It reduces resource consumption caused by repeated calculations and shortens query response time through parallel processing, effectively solving the performance bottleneck problem of large-scale data queries.

[0147] In some embodiments, a dynamic resource scheduling algorithm is used to allocate appropriate computing resources to each sub-query unit, intelligent caching management is implemented for intermediate results during execution, and a mapping relationship between query statement features and cached data is established, including:

[0148] Based on the computational complexity characteristics of subquery units and the data dependency graph, a multi-objective optimization algorithm is used to balance resource allocation among computing nodes.

[0149] The resource allocation strategy is dynamically adjusted based on the actual execution progress of each unit.

[0150] The intermediate result data generated during the execution process is valued, and the cache priority is determined based on the access popularity prediction model.

[0151] Construct a mapping index between query feature vectors and cached data blocks, and use a consistent hashing algorithm to achieve efficient location of distributed cache;

[0152] In addition, cached content is dynamically maintained based on data access frequency and freshness requirements.

[0153] In this embodiment, computational complexity is a quantitative assessment of the computational intensity, data size, and operation type of a subquery unit, while the data dependency graph describes the data flow and dependencies between subquery units. Based on these characteristics, a multi-objective optimization algorithm performs resource balancing among computing nodes, comprehensively considering multiple optimization objectives such as computational load balancing, data transmission cost, and execution efficiency to ensure optimal resource allocation.

[0154] The dynamic resource allocation strategy adjusts the allocation of computing resources such as CPU and memory in real time based on the actual execution progress of each unit and the system resource status. This strategy monitors execution indicators and resource utilization, and uses a feedback control mechanism to dynamically optimize resource allocation, ensuring the overall efficiency and stability of query execution.

[0155] Value assessment evaluates the importance of intermediate result data generated during the execution process. Based on the access popularity prediction model, it analyzes the probability of data being reused. This model determines the cache priority through historical access patterns and query characteristics analysis, prioritizing the caching of high-value intermediate result data to improve cache utilization efficiency.

[0156] A query feature vector quantifies the structural features, filtering conditions, and aggregation operations of a query statement into a numerical vector, while a cached data block is the storage unit for intermediate results. Building a mapping index can be understood as establishing a relationship between the query and cached data through feature matching, and using a consistent hashing algorithm to achieve efficient location and load balancing of cached data in a distributed environment.

[0157] Based on data access frequency and freshness requirements, cached data is automatically updated and evicted to dynamically maintain cache content. Frequently accessed data is retained first, while expired or infrequently accessed data is promptly cleaned up to ensure efficient use of cache space and data consistency.

[0158] This embodiment achieves optimal allocation of computing and cache resources through intelligent resource scheduling and cache management mechanisms. This not only ensures the high efficiency of query execution but also improves data reuse rate through intelligent caching, effectively solving key issues of resource utilization and performance optimization in a distributed environment.

[0159] In some embodiments, the original dataset is processed in real time according to a preset analysis model to generate structured indicator data with trend analysis, anomaly detection, or attribution inference results, including:

[0160] Perform data quality checks and missing value handling on the original dataset to ensure the integrity and accuracy of the input data;

[0161] Select the corresponding analysis model based on the business scope of the indicator definition information, including time series forecasting model, isolated forest anomaly detection model or SHAP attribution analysis model;

[0162] The preprocessed raw dataset is input into the selected analysis model to perform batch or streaming computation and generate preliminary analysis results.

[0163] The preliminary analysis results are post-processed and optimized, including result smoothing, confidence interval calculation, and significance labeling.

[0164] The optimized preliminary analysis results are linked and integrated with the original indicator data to generate structured indicator data that includes basic data and analysis conclusions.

[0165] In this embodiment, data quality verification is the process of verifying the integrity, consistency, and accuracy of the original dataset. Missing value handling includes identifying and filling in missing data values. These preprocessing operations are implemented through data exploration and statistical analysis methods to ensure the integrity and accuracy of the input data, providing a reliable data foundation for subsequent analysis.

[0166] The business definition of indicators refers to the definition specifications and calculation rules of indicators at the business level, and the selection of analytical models is based on matching these business requirements. Time series forecasting models are used for trend analysis, isolated forest anomaly detection models identify data anomalies, and SHAP attribution analysis models explain the causes of indicator fluctuations. The selection of these models ensures that the analysis results are highly consistent with business needs.

[0167] Batch or streaming computing refers to selecting different computing modes based on the data scale and timeliness requirements. Batch computing is suitable for historical data analysis, while streaming computing supports real-time data processing. The preprocessed raw dataset is input into the selected analysis model to perform calculations and generate preliminary analysis results that include trend prediction, anomaly labeling, or attribution analysis.

[0168] Post-processing optimization refers to further processing of the preliminary analysis results. This includes smoothing the results to eliminate random fluctuations, calculating confidence intervals to assess the reliability of the results, and using saliency markers to identify significant changes. These optimization operations are implemented through statistical methods and algorithms, improving the readability and usability of the analysis results.

[0169] The optimized preliminary analysis results are then structurally correlated with the original indicator data to generate a complete output containing basic data and analytical conclusions. This correlation and integration is achieved through data association technology and structured storage, resulting in structured indicator data that can be directly used for business decision-making.

[0170] This embodiment achieves automated transformation from raw data to intelligent analysis results through standardized analysis model application and systematic post-processing workflow. It not only ensures the standardization of the analysis process, but also enhances the business value of the data through multi-dimensional analysis, effectively improving the analytical depth and practical value of indicator data.

[0171] In some embodiments, filtering and de-identifying structured indicator data based on fine-grained access control strategies includes:

[0172] Parse the requester's identity credentials and access context, and extract permission attributes, including the user identifier, role permissions, and data access scope constraints corresponding to the requester.

[0173] Based on the extracted permission attributes, match them with a predefined set of access control rules to determine the scope of access the requester can have to the target metric data.

[0174] Based on the defined access scope, row and column level data filtering is performed on structured indicator data to remove data content outside the requester's permission scope;

[0175] Apply dynamic desensitization algorithms to sensitive fields in the retained data, and implement differentiated desensitization processing based on the field sensitivity level and user permissions;

[0176] Generate audit log records containing details of data filtering and de-identification, establish correlations with the processed indicator data, and construct standardized data response packages.

[0177] In this embodiment, the identity credentials and access context are the authentication information and access environment parameters provided by the requester. Permission attributes include key attributes such as user identifier, role permissions, and data access scope constraints. By parsing this information, the system can accurately identify the requester's identity characteristics and permission boundaries, providing a basis for subsequent access control decisions.

[0178] An access control rule set refers to a predefined set of data access permission rules. Based on extracted permission attributes, rule matching is performed to determine the scope of access that a requester can have to the target metric data. This matching is achieved through a rule engine, ensuring the accuracy and consistency of permission determination and effectively controlling data access boundaries.

[0179] Row- and column-level data filtering refers to fine-grained filtering of structured indicator data based on a defined access scope, removing data rows and columns outside the requester's authorized permissions. This filtering is achieved through SQL condition filtering or data processing, ensuring that users can only access data content within their authorized scope, thus achieving secure data isolation.

[0180] Dynamic data masking algorithms implement differentiated data masking processing based on field sensitivity levels and user permissions. They transform the content of sensitive fields in the retained data. Through techniques such as masking, hashing, or encryption, the algorithm protects sensitive information while ensuring data availability, achieving a balance between data security and usage needs.

[0181] Audit logs are operation records that include details of data filtering and anonymization, and are correlated with the processed metric data. Audit logs are automatically generated by the logging system, constructing standardized data response packages to provide complete evidence for data security audits and issue tracing.

[0182] This embodiment achieves refined security management of structured indicator data through multi-level access control and intelligent data processing mechanisms. It not only ensures the compliance of data access, but also ensures the traceability of data use through comprehensive audit trails, effectively resolving the contradiction between data security and sharing.

[0183] In some embodiments, the system returns the data content accessible within the requester's permissions and records the complete data access behavior to the audit log for tracing and analysis, including:

[0184] The standardized data response packet is returned to the requester through a secure transmission channel, and data fingerprint verification information is added to the response header of the standardized data response packet.

[0185] Capture the entire data access process in real time, including the requester's identity, access time, query conditions, returned data volume, and processing time.

[0186] The captured end-to-end behavioral information is associated and integrated with the corresponding indicator definition information and access control policies to generate structured audit log records.

[0187] The audit logs are encrypted and stored using blockchain technology.

[0188] In this embodiment, the secure transmission channel is a data transmission link established using encryption protocols such as TLS / SSL, used to securely return standardized data response packets to the requester. Data fingerprint verification information is a unique identifier generated using a hash algorithm, added to the response header to verify data integrity and prevent tampering, ensuring the secure and reliable transmission process.

[0189] End-to-end behavioral information refers to the complete operation record generated during data access, including key information such as the requester's identity, access time, query conditions, returned data volume, and processing time. By capturing this information in real time, the system can comprehensively monitor data access behavior, providing a complete raw data foundation for audit analysis.

[0190] The captured end-to-end behavioral information is logically correlated with corresponding indicator definitions and access control policies to generate structured audit log records. Through data association technology and structured storage, this correlation and integration is achieved, forming audit records containing complete contextual information, facilitating subsequent source tracing and analysis.

[0191] Blockchain-based evidence storage technology utilizes distributed ledger technology to encrypt and store audit logs, ensuring the immutability and traceability of log records through a hash chain structure and distributed consensus mechanism. Through encryption algorithms and distributed storage, blockchain-based evidence storage technology provides trusted audit evidence for data access activities.

[0192] This embodiment achieves controllability and traceability of the entire data access process through a combination of secure transmission, complete recording, and trusted evidence storage technologies. It not only ensures the security of data transmission but also guarantees the immutability of audit records through blockchain technology, providing reliable technical support for data security governance.

[0193] By adopting the above technical solutions, this invention differs from existing technologies and possesses the following beneficial effects: It achieves unified and standardized management of indicator definitions by constructing a global indicator asset library; automatically generates standardized query statements based on indicator-data mapping relationships; selects the optimal computing engine to execute queries using intelligent routing strategies; and improves query efficiency by employing multi-level caching optimization and parallel computing acceleration technologies. It processes raw data in real-time using a preset analysis model to generate structured indicator data; implements data filtering and de-identification processing based on fine-grained access control strategies; and finally returns the data through a secure transmission channel and uses blockchain technology to store audit logs. This method effectively solves problems such as inconsistent indicator definitions, severe duplication of development, low query performance, and insufficient data security control, achieving fully automated processing of indicator data from definition and production to application, significantly improving data production efficiency, quality consistency, and security reliability.

[0194] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0195] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0196] The above description is only a part of the embodiments of the present invention and does not limit the scope of protection of the present invention. Any equivalent device or equivalent process transformation made based on the content of the present invention specification and drawings, or direct or indirect application in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A data production and application method based on index management, characterized in that, The method comprises the following steps: receiving an index query request, the index query request comprising a target index identifier, dimension constraint conditions and time range parameters; analyzing and performing semantic verification on the target index identifier based on a global index asset library to obtain verified index definition information, the global index asset library storing definition information of a plurality of global indexes, each global index comprising an atomic index, a derived index and a composite index, and the definition information comprising a business scope, technical calculation logic, associated data sources and dimensions of the index; generating a standardized query statement adapted to the structure of a bottom-layer data warehouse based on the index definition information and the index-data mapping relationship, and selecting an optimal data calculation engine to execute the query according to an intelligent routing strategy; applying multi-level cache optimization and parallel computing acceleration to the query execution process to obtain a raw data set, the multi-level cache optimization comprising preheating storage and dynamic updating of high-frequency index query results, and the parallel computing acceleration adopting an MPP architecture for distributed processing; processing the raw data set according to a preset analysis model to generate structured index data with trend analysis, anomaly detection or attribution inference results; filtering and desensitizing the structured index data based on a fine-grained permission control strategy, returning only the data content accessible within the permission range of the requester to the requester, and recording the complete data access behavior to an audit log for traceability analysis; wherein the step of selecting an optimal data calculation engine to execute the query according to an intelligent routing strategy comprises: obtaining query statement features of the standardized query statement, the query statement features comprising syntax features, data volume estimation results and complexity scores; matching the query statement features with a preconfigured engine capability profile, the engine capability profile comprising performance and resource consumption characteristics of each data calculation engine for specific query types; comprehensively evaluating the query delay, resource utilization and calculation cost based on real-time load monitoring data through a multi-objective optimization algorithm to obtain an evaluation result; selecting an optimal target execution engine from an available engine cluster according to the evaluation result, and generating corresponding query task distribution instructions; establishing a session connection between the query task and the execution engine, and completing the initialization configuration of the query context environment.

2. The index management based data production and application method according to claim 1, wherein, The global index asset library is generated by the following steps: receiving an index registration request, the index registration request comprising the business scope, technical calculation logic, associated data sources and dimension information of the index to be registered; classifying the index to be registered into an atomic index, a derived index or a composite index according to the reuse level of the technical calculation logic; if the index to be registered is an atomic index, mapping and modeling it with the measurement field and its default aggregation method of a core fact table in the data warehouse; if the index to be registered is a derived index or a composite index, deriving and modeling it based on the already defined atomic index or derived index by configuring dimensions, filtering conditions and operation formulas; persisting the mapping relationship between the output index to be registered after modeling and the data warehouse table, field and calculation rule to form traceable index-data blood relationship, and forming the index definition information. Storing the indicator definition information into a global indicator asset library and assigning a globally unique versioned identifier to the indicator associated with the indicator definition information; Performing lifecycle management on the indicator definition information in the global indicator asset library, including automatic inspection and archiving based on access frequency and business relevance.

3. The indicator management based data production and application method according to claim 1, wherein, Analyzing and semantically verifying the target indicator identifier based on the global indicator asset library to obtain verified indicator definition information, including: Retrieving complete indicator definition information associated with the target indicator identifier from the global indicator asset library; Performing syntax compliance checking on the retrieved indicator definition information to verify the completeness and standardization of the business scope, technical calculation logic, associated data source, and dimension information of the retrieved indicator definition information; According to the preset indicator semantic rule library, performing semantic consistency verification on the indicator definition information that passes the syntax check to ensure that there is no contradiction between the business scope and the technical calculation logic; When detecting semantic ambiguity or conflict in the indicator definition information, triggering an indicator scope exception handling process to generate a verification report containing specific conflict items and correction suggestions; Applying a digital signature and a timestamp to the indicator definition information that passes all verifications to generate verified indicator definition information that can be trusted by subsequent query execution components.

4. The indicator management based data production and application method of claim 1, wherein, Generating a standardized query statement that adapts to the underlying data warehouse structure according to the indicator definition information through an indicator-data mapping relationship, including: Parsing the verified indicator definition information and extracting the technical calculation logic, the identifier of the associated data source, and the dimension information as query generation elements; Determining the corresponding data warehouse table, field name, and association based on the query generation elements through the indicator-data mapping relationship; According to the business scope and technical calculation logic of the indicator definition information, converting the dimension constraint conditions and time range parameters into corresponding SQL filter condition clauses; According to the data warehouse hierarchical model optimization principle, automatically selecting the optimal data level and constructing a complete query body containing the select list, group by grouping, and aggregation function; Using an abstract syntax tree-based query optimization algorithm to optimize the structure of the generated query statement, eliminating redundant calculations and ensuring syntax compliance with the target computing engine specification, and outputting an optimized and fully adapted standardized query statement to the underlying data warehouse structure.

5. The indicator management based data production and application method of claim 1, wherein, Applying multi-level cache optimization and parallel computing acceleration to the query execution process to obtain the original data set, including: Before query execution, perform cache matching retrieval based on query statement characteristics and historical execution records. If the cache is hit, return the cached data directly; For query requests that do not hit the cache, use a cache warming strategy based on a cost model to identify high-frequency access patterns and precompute and store intermediate results; Break down complex query tasks into multiple parallel-executable subquery units and generate an optimal parallel execution plan through a query optimizer; Use a dynamic resource scheduling algorithm to allocate appropriate computing resources to each subquery unit and implement intelligent cache management on intermediate results during execution to establish a mapping relationship between query statement characteristics and cached data; The final query result is compressed and stored, and the cache metadata information is updated to generate the original data set.

6. The indicator management based data production and application method according to claim 5, wherein, An appropriate computing resource is allocated to each subquery unit by using a dynamic resource scheduling algorithm, intelligent cache management is implemented for the intermediate results in the execution process, and a mapping relationship between query statement features and cached data is established, including: Based on the computing complexity features and data dependency graph of the subquery units, resource balancing allocation is performed among the computing nodes through a multi-objective optimization algorithm; The resource allocation strategy is dynamically adjusted according to the actual execution progress of each unit; The value of the intermediate result data generated in the execution process is evaluated, and the cache priority is determined based on an access frequency prediction model; A mapping index of query feature vectors and cache data blocks is constructed, and a consistent hashing algorithm is used to realize efficient positioning of the distributed cache; And the cache content is dynamically maintained according to the data access frequency and freshness requirements.

7. The indicator management based data production and application method of claim 1, wherein, The original data set is processed in real time according to the preset analysis model to generate structured index data with trend analysis, anomaly detection or attribution inference results, including: Data quality verification and missing value processing are performed on the original data set to ensure the integrity and accuracy of the input data; According to the business scope of the index definition information, the corresponding analysis model is selected, including a time series prediction model, an isolation forest anomaly detection model or a SHAP attribution analysis model; The preprocessed original data set is input into the selected analysis model to perform batch or streaming calculation, generating preliminary analysis results; The preliminary analysis results are post-processed and optimized, including result smoothing processing, confidence interval calculation and significance annotation; The optimized preliminary analysis results are associated and integrated with the original index data to generate structured index data containing basic data and analysis conclusions.

8. The indicator management based data production and application method of claim 1, wherein, The structured index data is filtered and desensitized based on a fine-grained permission control strategy, including: The identity credentials and access context of the requester are analyzed, and the permission attributes are extracted, including the user identifier, role permission and data access range constraint corresponding to the requester; The access control rule set is matched according to the extracted permission attributes to determine the access range of the target index data for the requester; Based on the determined access range, row and column level data filtering is performed on the structured index data to remove data content outside the requester's permission range; A dynamic desensitization algorithm is applied to the sensitive fields in the retained data to implement differential desensitization processing according to the field sensitivity level and user permission; Audit log records containing data filtering and desensitization processing details are generated, and an association relationship is established with the processed index data to construct a standardized data response package.

9. The indicator management based data production and application method according to claim 8, wherein, The data content accessible within the permission range of the requester is returned to the requester, and the complete data access behavior is recorded in the audit log for traceability analysis, including: The standardized data response package is returned to the requester through a secure transmission channel, and data fingerprint verification information is added to the response header of the standardized data response package; Real-time capture of full-link behavior information for this data access, including requester identity, access time, query conditions, returned data volume and processing time consumption; The captured full-link behavior information is associated and integrated with corresponding index definition information and permission control strategies to generate structured audit log records; The audit log is stored by using a blockchain storage technology.

Images