A metadata-driven report compliance generation method and system

Abstract

The application discloses a kind of report compliance generation method and system based on metadata driving, belong to data processing technical field.The method includes: the original metadata of heterogeneous database is collected, is converted into unified metadata model;In response to report configuration operation generation semi-structured SQL template;In response to report query request, build security context, execute SQL after injecting permission condition and obtain query result set;In response to data entry operation, load compliance rule, and check business data object by rule engine;When check does not satisfy and is configured as automatic silent execution mode, execute correction in memory, and write the correction data into business table in the same database transaction, simultaneously change context is written into audit log table;In response to export request, stream generation data file.The application realizes automatic compliance correction, and guarantees the consistency of rule in heterogeneous database environment based on unified metadata model by atomicity audit, and efficiency and compliance are considered.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, specifically relating to a metadata-driven method and system for generating compliant reports. Background Technology

[0002] Current enterprise-level reporting systems have developed relatively mature technical solutions for data output. For example, CN121597682A discloses a method and system for dynamic data export based on template configuration. This method achieves efficient export of large amounts of data by driving dynamic SQL generation, paginated queries, and streaming writes through template metadata. It also employs an RBAC model for access control and supports template version management. This solution effectively addresses the issues of flexibility, efficiency, and security in the report generation and export process.

[0003] However, current reporting systems, including the aforementioned existing technologies, primarily focus on the data output stage (i.e., how to efficiently and flexibly query and export data from data sources), while systematically lacking in the subsequent data quality governance stages such as data input, supplementation, and correction. Specifically, this manifests in the following technical issues: First, there is a lack of automated compliance correction capabilities for entered data. Existing technologies only implement a "verification-error reporting" model or rely on manual judgment for data correction. When the business scale expands and the frequency of data entry increases, manual correction becomes an efficiency bottleneck, making it impossible to achieve intelligent decision-making and automatic execution based on business rules.

[0004] Second, there is a contradiction between automatic correction and compliance auditing. If fully automatic correction is used, the correction behavior lacks a reliable audit mechanism, resulting in inconsistencies such as "data has changed but there is no record" or "records exist but the data has not changed," which cannot meet financial-grade audit requirements. If manual correction is relied upon, it is inefficient. Existing technologies have failed to solve the technical challenge of simultaneously prioritizing both "efficiency" and "security and compliance."

[0005] Third, there is a lack of unified rule consistency guarantees in heterogeneous database environments. Existing technologies do not consider the difficulty in uniformly expressing and executing data compliance rules (such as data masking and format validation) across data sources in hybrid database architectures such as GaussDB, GoldenDB, MySQL, and PostgreSQL. This results in the same business rule needing to be implemented in different databases, leading to fragmentation of business rules.

[0006] In summary, existing technologies focus on optimizing efficiency and flexibility in the report output process, but have not yet built a complete technological closed loop that covers the entire data lifecycle and balances the efficiency of automated correction with compliance and traceability. Summary of the Invention

[0007] This invention aims to overcome the shortcomings of existing technologies and provide a metadata-driven method and system for generating compliant reports. It addresses the technical problems of existing reporting systems, such as the lack of automated compliance correction capabilities during data entry, the difficulty in achieving both automatic correction and compliance auditing, and poor rule consistency in heterogeneous database environments. This invention enables compliance governance of report data throughout its entire lifecycle, from definition, generation, correction to auditing.

[0008] To achieve the above objectives, this invention provides a metadata-driven report compliance generation method, comprising the following steps: Step 1: Metadata Standardization Mapping Connect to at least one heterogeneous database using a pluggable adapter to collect raw metadata from each database; convert the collected raw metadata into a standardized unified metadata model according to a preset unified type mapping rule; each field object in the unified metadata model includes at least a field name, a field type, and a bound compliance rule identifier.

[0009] Step 2: Dynamic Configuration and SQL Template Generation In response to a report configuration operation, the system obtains the set of fields selected by the user from the unified metadata model and the configured query conditions; based on the set of fields and the query conditions, it generates a semi-structured SQL template through a declarative compilation pipeline; the semi-structured SQL template contains a fixed SQL structure and at least one permission injection placeholder.

[0010] Step 3: Runtime Permission Binding In response to a report query request, the user identity information in the request is parsed to construct a security context object; based on the security context object and the permission policy corresponding to the semi-structured SQL template, permission filtering conditions are generated; the permission filtering conditions are injected into the permission injection placeholder of the semi-structured SQL template to generate an executable SQL statement; the executable SQL statement is executed to obtain a query result set.

[0011] Step 4: Data entry and compliance verification In response to the entry or correction of report data, obtain the business data object to be persisted; based on the compliance rule identifier bound to the field object in the unified metadata model, load the corresponding compliance rule from the rule library; and use the rule engine to verify the values ​​of each field in the business data object to determine whether they meet the compliance rule.

[0012] Step 5: Smart Correction and Atomicity Audit When the verification judgment is not met, if the compliance rule is configured to automatic silent execution mode, the rule engine performs a correction operation on the corresponding field value in the business data object in memory to generate a corrected business data object; constructs a change context object, which at least records the original value, the corrected value, and the triggered rule identifier of the corrected field; in the same database transaction, writes the corrected business data object into the business database table, and serializes the change context object and writes it into the audit log table; in response to the successful commit of the transaction, returns a data entry success response.

[0013] Step 6: Output the result set In response to a report export request, the system retrieves the query result set or the corrected business data object; it then generates a data file in a streaming manner according to a preset export format and returns a download link.

[0014] To implement the above method, the present invention also provides a metadata-driven report compliance generation system, the system comprising: The metadata mapping module is used to connect to at least one heterogeneous database through a pluggable adapter, collect the raw metadata of each database, and convert the collected raw metadata into a standardized unified metadata model according to a preset unified type mapping rule; each field object in the unified metadata model includes at least a field name, a field type, and a bound compliance rule identifier.

[0015] The configuration management module is used to respond to report configuration operations, obtain the set of fields selected by the user from the unified metadata model and the configured query conditions, and generate a semi-structured SQL template through a declarative compilation pipeline based on the set of fields and the query conditions; the semi-structured SQL template contains a fixed SQL structure and at least one permission injection placeholder.

[0016] The execution engine module is used to respond to report query requests, parse the user identity information in the request, construct a security context object; generate permission filtering conditions based on the security context object and the permission policy corresponding to the semi-structured SQL template; inject the permission filtering conditions into the permission injection placeholder of the semi-structured SQL template to generate an executable SQL statement; and execute the executable SQL statement to obtain a query result set.

[0017] The compliance correction module further includes: The rule base unit is used to store compliance rules. Each compliance rule includes at least a rule identifier, rule type, conditional expression, execution action, and execution mode. The execution modes include automatic silent execution mode and suggested execution mode. The real-time verification unit is used to respond to the entry or correction operation of report data, obtain the business data object to be persisted, load the corresponding compliance rule from the rule base unit according to the compliance rule identifier bound to the field object in the unified metadata model, and verify the field values ​​in the business data object through the rule engine. The intelligent correction unit is used to perform a correction operation on the corresponding field value in the business data object in memory when the judgment result of the real-time verification unit is not satisfied. If the current compliance rule is configured to be in automatic silent execution mode, the unit will generate a corrected business data object and construct a change context object. The change context object records at least the original value of the corrected field, the corrected value, and the rule identifier that was triggered. An atomic audit unit is used to write the corrected business data object into a business database table and serialize the change context object into an audit log table within the same database transaction, and to return a data entry success response in response to the successful commit of the transaction.

[0018] The file generation module is used to respond to the report export request, obtain the query result set or the corrected business data object, and generate a data file in a streaming manner according to a preset export format and return a download link.

[0019] Compared with existing technologies and common knowledge in the field, the present invention has the following significant beneficial technical effects: I. The effect of technology integration based on the unified metadata model 1. Eliminate the data semantic gap between the output and input stages. In existing technologies, template metadata only serves the SQL generation and data export stages, and there is no technical connection between it and the validation rules in the data entry stage. This results in the need to define and maintain the desensitization rules for the same field in report output and the validation rules in data entry separately, which has an inherent defect of inconsistent rules.

[0020] In step 1, this invention maps the raw metadata of heterogeneous databases to a unified metadata model containing field types and compliance rule identifiers. In step 4, the corresponding compliance rules are loaded and validated based on the compliance rule identifiers bound to this unified metadata model. This technique enables a unified expression of the semantic definition, output format rules, and input validation rules for the same field at the data structure level, fundamentally eliminating the inconsistency between rules in the output and input stages. This effect is not a simple addition of metadata collection and rule binding, but rather achieves decoupling and re-unification of rule definition and rule execution by storing and transmitting rule identifiers as inherent attributes of metadata.

[0021] 2. Achieve consistent rule execution across heterogeneous databases Existing technologies do not take into account heterogeneous database environments. Their template metadata is directly bound to the fields and dialects of specific databases, making it impossible to reuse the same rule definition across different databases such as GaussDB, MySQL, and PostgreSQL.

[0022] In step 1 of this invention, raw metadata from each database is collected via a pluggable adapter and converted into standardized types according to a preset unified type mapping rule (e.g., mapping VARCHAR, CHAR, and TEXT types from each database to String type). This technique binds compliance rules (such as mobile phone number anonymization rules) to standardized field objects rather than the raw types of specific databases. Therefore, when the rule engine performs validation in step 4, it does not need to be aware of the type differences in the underlying databases, thus achieving consistent execution of the same rule in heterogeneous database environments. This effect relies on the synergy of two sub-methods: the adapter solves the problem of "how to acquire," and the type mapping solves the problem of "how to uniformly express," and neither is dispensable.

[0023] II. Technological Breakthrough Effects Based on Intelligent Correction and Atomic Auditing 1. For the first time, it resolves the technical contradiction between "automatic correction efficiency" and "compliance traceability". In existing technologies and common knowledge in the field, the quality check of entered data typically adopts a "verification-error reporting-manual correction" model, or an automatic correction model where audit records are separated from the correction operation. The former is inefficient, while the latter carries the risk of inconsistencies such as "data has changed but audit is missing" or "audit records exist but data has not changed."

[0024] In step 5, this invention resolves the aforementioned contradiction through a combination of the following technical means: First, the rule engine performs a correction operation on the business data object in memory, avoiding the overhead of multiple database accesses; second, a change context object is constructed to fully record the original value, the corrected value, and the triggered rule identifier; finally, within the same database transaction, the serialized results of the corrected business data and the change context object are written to the business database table and the audit log table, respectively. This nested write architecture within a transaction ensures atomicity in the persistence of business data and audit records—both operations either commit successfully simultaneously or roll back simultaneously, ensuring strong consistency at the database transaction level: "Every correction must have a record, and every record must correspond to a real correction." This effect relies on the sequential coordination of the three technical means: memory correction, change context construction, and nested write within a transaction; the absence of any one of these steps will prevent the achievement of this effect.

[0025] 2. Enable the legitimate use of automatic silent execution mode in financial-grade scenarios. Under the existing technical architecture, due to the lack of audit atomicity guarantees, automatic correction cannot pass financial compliance audits, forcing the system to adopt a manual confirmation mode, resulting in user experience interruption and decreased operational efficiency.

[0026] This invention provides an auditable, traceable, and tamper-proof technical foundation for the automatic silent execution mode through the intra-transaction atomic write mechanism in step 5. The audit log table adopts an append-only storage strategy and a hash chain structure, with each audit record containing the hash value of the previous record, forming an immutable chain of evidence. This allows the automatic silent execution mode, which was originally excluded due to compliance risks, to be legally enabled while meeting financial-grade audit requirements, achieving a balance between efficiency and security. This effect relies on the synergy of two technical means: the atomic write in step 5 and the tamper-proof storage of the audit log; the former ensures "record exists," and the latter ensures "records are non-repudiable."

[0027] III. Optimization Effects Based on Layered Compilation and Dynamic Injection 1. Decouple and reuse permission policies and SQL templates. In existing technologies, access control uses the RBAC model for pre-configuration at the template level. The permission conditions are fixed in the SQL statement when the template is defined, and cannot be dynamically adjusted according to the request context. The same template cannot serve user requests with different permission granularities.

[0028] This invention achieves a technological breakthrough through the layered collaboration of steps 2 and 3: Step 2 generates a semi-structured SQL template containing permission injection placeholders through a declarative compilation pipeline, thus separating permission logic from the fixed SQL structure; Step 3 generates permission filtering conditions at runtime based on a security context object (containing dynamic information such as user roles, organization IDs, and request parameters), and injects placeholders. This layered architecture of "generating the skeleton at compile time and injecting conditions at runtime" allows the same semi-structured SQL template to serve user requests at different permission levels, and changes to permission rules do not require regenerating the SQL template. This effect relies on the sequential collaboration of three technical means: pre-setting placeholders, dynamically constructing the security context, and runtime injection.

[0029] 2. Achieve dual-layer collaborative optimization of real-time front-end validation and intelligent back-end correction. Existing technologies do not involve a collaborative design between front-end validation and back-end correction. All validation logic is executed on the back-end, and invalid requests will still consume database connections and computing resources.

[0030] The present invention implicitly employs a two-layer validation architecture in the collaboration between steps 4 and 5: In step 4, the rule engine stratifies the validation task based on rule type—lightweight rules corresponding to format-based and required-field rules are executed immediately on the front end, reducing network transmission of invalid requests and backend computation; complex logic validation and correction decisions are executed in the backend rule engine. Both layers of validation share the same set of rule definitions bound to the unified metadata model in step 1, achieving a reasonable division of validation responsibilities through the differentiation of rule type fields. This effect relies on the collaboration of three technical features: predefined rule types, rule distribution mechanisms, and consistent execution of rules on both the front end and back end.

[0031] IV. Overall Evaluation of Technical Effects The aforementioned technical features of this invention do not function independently, but rather, through a unified metadata model as a coherent technical link, atomic writes within transactions as a consistency guarantee, and layered processing at compile time and runtime as the architectural foundation, they produce a synergistic technical effect that transcends the individual contributions of each feature: Synergy in efficiency: Metadata standardization in step 1 eliminates the need for reconfiguration in rule validation in step 4; memory correction in step 5 avoids secondary processing of the original data in the export stage in step 6; and real-time front-end validation reduces invalid calculations by the back-end rule engine—these three factors combined achieve end-to-end efficiency improvements.

[0032] Collaboration in security dimensions: The dynamic permission binding in step 3 ensures that the data entered in step 4 is always within the permission framework; the atomicity audit in step 5 ensures that the data exported in step 6 has a complete change traceability chain—security policies run through the entire process rather than existing in isolation.

[0033] Coordination in consistency: The unified metadata model in step 1 provides the same semantic basis for SQL generation in step 2, rule validation in step 4, and correction actions in step 5, ensuring that the same field is treated consistently in all stages.

[0034] Furthermore, a common technical bias exists in the art that "automatic correction is difficult to meet compliance audit requirements," leading related systems to often sacrifice automation to ensure traceability, or sacrifice traceability to achieve automation efficiency. This invention, through the technical architecture design of "memory correction → context change construction → nested write within transactions" in step 5, demonstrates in an engineering manner that efficiency and compliance can be achieved simultaneously, overcoming this long-standing technical bias.

[0035] In summary, this invention, through the organic synergy of steps 1 to 6, constructs a complete technical closed loop encompassing metadata standardization, dynamic configuration, runtime permission binding, intelligent correction, and atomic auditing. This achieves superior technical results compared to existing technologies in multiple dimensions, including efficiency improvement, security compliance, and cross-platform consistency. These results are not simply the sum of the functions of each step, but rather stem from the qualitative technological advancements resulting from the deep integration and collaborative operation of these steps based on technologies such as a unified metadata model, atomic writes within transactions, layered compilation, and dynamic injection. Attached Figure Description

[0036] Figure 1 : Overall flowchart of the method described in Example 1; Figure 2 Example 1: Step 1 flowchart; Figure 3 Example 1, Step 2 flowchart; Figure 4 Example 1, Step 3 flowchart; Figure 5 Example 1, Step 4 flowchart; Figure 6 Example 1, Step 5 flowchart; Figure 7 Example 1, Step 6 flowchart; Figure 8 Example 2: Overall architecture diagram of the system described. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in further detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of protection of this invention.

[0038] In the following description, for ease of understanding, some of the technical terms involved in this invention are explained as follows: Semi-structured SQL: refers to a SQL template form that separates fixed logic from variable logic. It is not a complete SQL statement that can be executed directly, but a "skeleton" SQL representation that includes a predefined structure (such as a list of SELECT fields, FROM table name, etc.) and specific placeholders (used for dynamic parameters and permission rule injection).

[0039] Permission hooks: These are specific flags or placeholders (such as "${AUTH_FILTER}") preset in semi-structured SQL templates, used to instruct the runtime engine to precisely inject dynamically generated permission filtering conditions at that location.

[0040] Security context: refers to the data object that the system builds throughout the entire chain after intercepting the HTTP request and parsing the user's credentials (such as a token) when the user initiates a report request. This object encapsulates the identity attributes of the current operator (user identifier, role set, affiliated organization identifier, data permission scope) and environmental information (request parameters, current timestamp, client IP address).

[0041] Streaming generation: refers to a technical strategy of querying, processing, and writing data simultaneously when exporting report query result sets to files, rather than loading all data into memory at once and then writing it to the file, in order to reduce peak memory usage.

[0042] Structured policy objects refer to permission or compliance rules expressed in a standardized JSON format. They include structured fields such as rule identifier, conditional expression tree, execution action, priority, and effective time range, and support machine parsing and version management.

[0043] Change context: refers to a structured object built during the intelligent correction process that fully records the facts of data changes, including the original value, the corrected value, the correction rule, the execution mode, the timestamp, the session information, etc., and serves as the core data source for audit logs.

[0044] Automatic silent execution mode: refers to an execution strategy of the intelligent corrector. When the rule conditions are met, the system automatically completes the data correction without the user's awareness, without interrupting the user's operation process, and at the same time, the traceability of the correction record is guaranteed through transaction atomicity.

[0045] Atomicity auditing refers to placing the persistence operation of business data and the persistence operation of the audit record of the data change within the same database transaction boundary. By leveraging the atomicity of database transactions, it ensures that the two operations are either successfully committed or rolled back simultaneously.

[0046] WORM storage: refers to WriteOnceReadMany storage mode, which sets up a protection mechanism at the database layer or storage system layer that prevents modification and deletion after writing.

[0047] Example 1 This embodiment provides a metadata-driven method for generating compliant reports. This method can be deployed on an enterprise-level application server and interact with one or more heterogeneous databases such as GaussDB, MySQL, PostgreSQL, and GoldenDB. It provides full lifecycle governance of report data for business scenarios with high requirements for data quality and compliance, such as finance and government.

[0048] Reference Figure 1 , Figure 1This is a flowchart illustrating the overall process of the method of the present invention, showing the complete flow of steps 1 to 6 and their sequential relationship. The following is in conjunction with the appendix... Figure 2 To be continued Figure 7 Each step is explained in detail.

[0049] Step 1: Metadata Standardization Mapping like Figure 2 As shown, Figure 2 This is a detailed flowchart of this step, which includes: When the system starts up or in response to a metadata collection command triggered by the administrator, it connects to each target heterogeneous database through a pre-configured pluggable database adapter.

[0050] The adapter framework provides pluggable database adapters, each encapsulating the database-specific system table query dialect. For GaussDB, the adapter uses the JDBC DatabaseMetaData interface combined with system table queries to collect raw metadata information such as table names, field names, field types, field lengths, nullability, primary key identifiers, and foreign key relationships from system tables like pg_tables, pg_attribute, and information_schema. For GoldenDB, the adapter performs distributed metadata aggregation through a dedicated management interface, uniformly gathering metadata from various shards. For MySQL, the adapter uses the JDBC DatabaseMetaData interface to collect the corresponding raw metadata from the information_schema.tables and information_schema.columns system views. For PostgreSQL, the adapter uses JDBC DatabaseMetaData combined with system functions to collect metadata from the pg_catalog system list.

[0051] After data collection, the system standardizes the original types according to preset unified type mapping rules. A unified type mapping table is established: GaussDB's VARCHAR, CHAR, and TEXT types, GoldenDB's charactervarying and text types, and MySQL's varchar and nvarchar types are uniformly mapped to the standardized type "String"; the int, bigint, and number types from various databases are uniformly mapped to the standardized type "Numeric".

[0052] Meanwhile, the system obtains primary key information through the DatabaseMetaData.getPrimaryKeys() method and foreign key relationships through the getExportedKeys() and getImportedKeys() methods, thus constructing a relationship graph between tables.

[0053] The system further performs automatic supplementation of business tags, which is based on the business tag mapping rule base. Each rule in the rule base contains the following elements: Rule ID (globally unique identifier, such as RULE_PII_PHONE_001), Triggering conditions (a combination of field metadata features, such as field_nameLIKE'%Mobile%' OR field_nameLIKE'%Phone%' OR field_commentLIKE'%Contact Information%'), Matching pattern (regular expression or semantic keyword library, such as exact match for "Mobile Number" or "Mobile Phone", fuzzy match for "Phone", "Contact Information", or "Tel"), Tags to be added (a set of standardized business tags and attributes, such as tag type is sensitivity level, tag value is PII, de-identification rule is to keep the first three and last four digits of the mobile number mask, compliance rules are R_001 and R_002), Priority weight (the basis for adjudication when multiple rules conflict, the smaller the value, the higher the priority, such as the priority of the exact match rule is 10, the priority of the fuzzy match rule is 50, and the priority of the default rule is 100), and Scope of application (the database type to which the rule applies, such as MySQL, PostgreSQL, GaussDB, or global GLOBAL).

[0054] The system loads the rule base into a memory cache (such as Redis), creates an index based on priority, and simultaneously builds an inverted index with keywords as keys and a list of rule identifiers as values ​​to accelerate matching. For each collected field object (FieldMetadata), the field name, field type, and field comment are extracted as matching keys. The inverted index is queried to obtain a list of candidate rules, and the triggering conditions of each rule (regular expression matching or keyword inclusion) are evaluated in turn. All matching rules are collected and deduplicated according to priority (high-priority tags cover low-priority conflicting items). The final set of business tags is written to the rule binding attribute of the field object. For example, when the field name is "mobile_phone" or the field comment contains "phone number", the tag "Sensitivity Level: PII, Desensitization Rule: Phone Number Mask (keeping the first three and last four digits)" is automatically added. This automated tag supplementation process can be completed in seconds for data sources containing thousands of fields.

[0055] The system also supports dynamic updates: the management interface supports adding, deleting, and modifying rules. After a change is made, it is hot-loaded to the cache of all nodes within 30 seconds via the RedisPub / Sub mechanism, without the need to restart the service, ensuring that the rules take effect in real time.

[0056] Finally, the system integrates the above processing results into a standardized unified metadata model. Each field object in this model contains at least a field identifier, field name, standardized field type, table identifier, primary and foreign key markers, and a list of bound compliance rule identifiers.

[0057] Step 2: Dynamic Configuration and SQL Template Generation like Figure 3 As shown, Figure 3 The detailed flowchart for this step illustrates the four stages of the declarative compilation pipeline.

[0058] Administrators create or modify report configurations through a visual configuration interface. This interface displays a list of selectable fields in the unified metadata model built in step 1, as well as controls such as query condition configurators, sort configurators, and group configurators.

[0059] This step uses a declarative compilation pipeline to convert configuration into an SQL template, which consists of four stages.

[0060] Phase one involves front-end configuration data collection. Responding to administrator actions such as field selection, filtering (e.g., selecting a time range for a date field, setting fuzzy matching keywords for a text field), sorting (specifying ascending or descending order), and grouping, the front-end converts these configurations into an Abstract Syntax Tree (AST). The ASTNode structure includes node types (nodeType, with enumerated values ​​including FIELD, FILTER, GROUP, SORT, and AGGREGATION), property sets (properties, a Map storing key-value pairs), a list of child nodes (children), and associated UI control identifiers (sourceRef). The mapping rules from UI controls to AST nodes are as follows: Field checkboxes are mapped to FIELD nodes, with attributes including fieldId, alias, and displayName; dropdown selectors are mapped to FILTER nodes, with operator IN and values ​​representing the list of selected items; date range selectors are mapped to FILTER nodes, with operator BETWEEN and values ​​including startDate and endDate; text input boxes are mapped to FILTER nodes, automatically recognizing the operator based on the input (pure numbers are mapped to EQUALS, and those containing % are mapped to LIKE); group drag-and-drop operations are mapped to GROUP nodes, with fields representing the list of group fields; sort buttons are mapped to SORT nodes, containing field and direction (ASC or DESC).

[0061] Phase two involves DSL parsing and IR generation. The system performs semantic validation on the generated AST, checking the existence of referenced fields in the unified metadata model and the compatibility of field types and operators (e.g., date type fields cannot use numeric comparison operators). After passing the validation, the system performs predicate pushdown optimization, marking conditions that can be executed at the database layer (e.g., single-table field filtering) and conditions that need to be processed at the application layer (e.g., calculated field filtering after cross-table joins), and scans for values ​​containing the syntax "${...}" in properties, extracting them as IRParameters and marking their types (USER_CONTEXT user context, REQUEST_PARAM request parameters, SYSTEM_VAR system variables). The system then converts the AST into an intermediate representation (IR). The IRStatement structure includes: selectList (an array of field objects, containing sourceField, alias, aggregation function, and dataType), whereClause (a tree of conditional expressions, with the root node being the logical operator AND or OR, and the leaf nodes being Condition objects), groupBy (a list of grouping field references), orderBy (a list of sorting objects, containing fields and direction), and parameters (a list of dynamic parameter placeholders). Notably, whereClause uses a tree structure rather than a flat list, supporting arbitrary levels of AND / OR nesting to express complex business logic such as "(AANDB)OR(CANDD)".

[0062] Phase three involves target code generation. The system converts IR into a parameterized semi-structured SQL template. The conversion rules include: field mapping, mapping IRField.sourceField to database dialect column names based on metadata (e.g., customerName maps to t1.customer_name); condition expression conversion, converting Condition objects into SQL fragments "{field}{operator}{placeholder}", mapping operators to EQUALS → "=", GREATER_THAN → ">", IN → "IN(?,?,...)", and BETWEEN → "BETWEEN?AND?"; logical combination expansion, recursively traversing the IREExpressionTree, generating AND nodes in the form "(condition1ANDcondition2)" and OR nodes in the form "(condition1ORcondition2)", supporting arbitrary levels of nesting; parameterization, replacing all dynamic values ​​with "?" placeholders, storing the original values ​​in the parameters list to prevent SQL injection attacks; and pre-setting a permission injection placeholder "${AUTH_FILTER}" at the end of the WHERE clause in the SQL template for subsequent injection of dynamic permission conditions.

[0063] Phase four involves execution plan caching. The system calculates the MD5 hash value of the normalized IR (removing specific parameter values ​​related to user sessions and retaining only the structure) as the cache key, and stores the generated semi-structured SQL template in the cache. The cache item structure is a mapping from the cache key (MD5 hash) to the compiled SQL template, supporting an LRU eviction policy. Queries with different parameters of the same structure reuse the same SQL template, requiring only rebinding of parameter values; when template metadata changes, the relevant cache is invalidated through a version number mechanism.

[0064] Step 3: Runtime Permission Binding like Figure 4 As shown, Figure 4 The detailed flowchart for this step illustrates the complete process of permission policy retrieval, condition generation, and injection.

[0065] When business users access reports through the client, the client sends a report query request to the server with user credentials (such as JWTToken).

[0066] The server intercepts the request, parses the user identity information in the token, and constructs a security context object. This security context object contains at least: user identifier (userId), user role list (roles), affiliated organization identifier (branchId), data permission scope (such as the accessible organization tree), request parameters (request, such as query condition values ​​passed by the front end), and system variables (system, such as the current timestamp and system configuration).

[0067] The system retrieves a set of effective permission policies from the policy store based on the report identifier of the current request, the user role in the security context, and the current time. Permission policies are stored as structured policy objects (JSON format), containing a policy identifier (policyId), policy name (policyName), policy type (policyType, such as ROW_FILTER row-level filtering), target resource (targetResource), target resource type (targetResourceType), rule expression (ruleExpression), join context (joinContext), priority (priority), effective time range (effectiveTimeRange), and status (status). The rule expression uses a tree structure, supports AND / OR logic combinations, and the condition node includes a condition identifier (conditionId), field name (field), operator (operator, EQUALS, IN, BETWEEN, LIKE, etc.), value reference (valueRef, which can be in SpEL expression form, such as "${user.branchId}" to retrieve the user's organization identifier from the security context), value type (valueType), required (required), and default value (defaultValue). Regular expressions also support subgroups, each containing a group operator (OR or AND), a list of conditions, and the target table identifier (appliesTo). The joinContext contains the main table alias (mainTableAlias) and joinMappings.

[0068] The system uses the SpEL expression engine to evaluate value references in the rule expression tree, supporting nested attribute access (e.g., "${user.org.parentId}") and method calls (e.g., "${user.hasRole('ADMIN')}"). It replaces "${user.branchId}" with the actual organization identifier of the current user and "${request.startDate}" with the start date from the request parameters. After evaluation, the system uses a recursive generation algorithm to convert the rule expression tree into database dialect-compatible SQL condition fragments. The algorithm logic is as follows: If the node type is AND or OR, the child nodes are processed recursively, and the SQL fragments generated by the child nodes are connected using node operators and enclosed in parentheses; if the node type is CONDITION, the next parameter placeholder is obtained, and the condition is formatted according to the operator mapping rules (EQUALS is mapped to "=", IN is mapped to "IN(?,?,...)", BETWEEN is mapped to "BETWEEN?AND?", LIKE is mapped to "LIKE?", and wildcard processing is automatically added); if the node type is SUBGROUP, the expressions of the subgroup are processed recursively, and comments are added to mark the target table to which the condition is applied. For multi-table join scenarios, the joinContext field of the strategy object specifies the table alias to which the condition is applied. An example of the generated condition is "t1.branch_id=?AND t1.data_dateBETWEEN?AND?AND(t2.region_code=?OR t2.region_codeISNULL)".

[0069] The system locates the permission injection placeholder "${AUTH_FILTER}" in the semi-structured SQL template generated in step 2, and precisely replaces this placeholder with the SQL fragment for permission filtering conditions generated after conversion. If no permission policy is matched, the placeholder is replaced with the tautological condition "1=1". The placeholder position index (character offset) is recorded during the pre-compilation stage, and the strings are directly concatenated at runtime (parameterized, eliminating SQL injection risk). After replacement, the dynamic parameter values ​​are bound to the placeholders in the pre-compiled statement to form a complete executable SQL statement, which is then submitted to the database for execution to obtain the query result set.

[0070] Step 4: Data entry and compliance verification like Figure 5 As shown, Figure 5 The detailed flowchart for this step illustrates the dual verification architecture of front-end real-time verification and back-end final verification.

[0071] Business personnel fill in data and submit it on the data entry page attached to the report. The server receives a request containing business data to be persisted, which is transmitted in the form of a Data Transfer Object (DTO) or an Entity.

[0072] This step employs a dual verification architecture of "real-time front-end feedback + final back-end backup".

[0073] The front-end real-time validation is triggered when user input becomes blurry or before form submission. The validation scope includes format (regular expression validation), length, required fields, and simple ranges (numerical upper and lower limits). Technically, the backend pre-deploys lightweight rule JSON (containing only rule types and simplified rule scripts, averaging less than 5KB) when the page loads. The frontend performs validation based on EXTJS 4.0 custom directives (v-compliance) or asynchronous validation components, supporting debouncing (300ms latency). The rule JSON is compressed for transmission, ensuring zero network latency for local execution.

[0074] The backend final validation intercepts methods annotated with compliance check annotations (such as @DataComplianceCheck) via Spring AOP aspects, triggering before business logic execution. The system obtains parameter types through method signatures, identifies data objects annotated with @Entity, and retrieves the list of compliance rule identifiers bound to each field object from the unified metadata model built in step 1 based on the report identifier corresponding to that business data object.

[0075] The system loads complete compliance rule definitions from the rule base based on rule identifiers. Compliance rules employ a two-way association model of "metadata anchors + policy registry." Metadata anchors are field objects in the unified metadata model, serving as the carrier for rule binding. The FieldMetadata extended attributes include boundRuleIds (a list of bound rule identifiers), inheritedRules (default rules inherited from the data type), and ruleEffectiveTime (the effective time window for each rule). The policy registry independently stores rule definitions, supporting version management and canary releases. Each compliance rule must include at least the following: rule ID (ruleId), rule name (ruleName), rule version (ruleVersion, supporting semantic version numbers), rule type (ruleType, enumerated values ​​include FORMAT format validation, RANGE range validation, LOGIC logic validation, ASSOCIATION association validation, TRANSFORM transformation correction), rule script (ruleScript, such as the regular expression "^[1-9]\d$" representing a positive integer or SpEL expression), pre-compiled execution object (compiledScript, used for cache reuse), priority (the smaller the value, the higher the priority, used for conflict resolution), mutex group ID (mutexGroup), execution mode (executionMode, AUTO_SILEN automatic silent execution mode or SUGGEST recommended execution mode), list of applicable data sources (applicableDataSources), effective time range (effectiveRange), and status (DRAFT draft, ACTIVE effective, DEPRECATED deprecated).

[0076] The rule binding process is as follows: Administrators select rules for fields in the visual interface. The system queries the policy registry to obtain the current version identifier of the selected rule, adds the rule identifier to `boundRuleIds` in `FieldMetadata`, synchronously updates the metadata cache, triggers rule preloading, and records the binding operation log (operator, time, rule version snapshot). When the process takes effect at runtime, the engine automatically retrieves the rule definition corresponding to `boundRuleIds` when loading report metadata, building a field-rule mapping cache (`FieldRuleCache`) to avoid repeated queries at runtime.

[0077] The backend validation uses a rule engine optimized based on the Rete algorithm. This algorithm constructs a rule condition network, with the Alpha network used to filter constant conditions and the Beta network used to handle variable binding. It shares the evaluation results of common conditions and achieves O(1) constant-level matching efficiency when binding 5 to 10 rules for a single field. The rule engine supports nested condition evaluation and complex Boolean expressions such as IF(AAND(BOR(CANDD)))THEN... It uses recursive descent parsing to construct an expression tree and post-order traversal for evaluation. For handling multiple rule conflicts, a priority adjudication plus mutual exclusion group mechanism is adopted: the smaller the priority value, the higher the priority. Only one rule with the same mutual exclusion group identifier is executed, and the higher priority rule is overridden in case of conflict. When rules have the same priority and are in the same mutual exclusion group, a RuleConflictException is thrown for manual handling. In terms of execution strategy, the SUGGEST suggestion mode pop-up window displays the correction suggestion and executes it after user confirmation. The AUTO_SILENT automatic silent mode executes it directly when the conditions are met, without the user's awareness.

[0078] The system performs rule validation on each field value in the business data object sequentially. For format validation rules (such as regular expression validation of mobile phone numbers), the field values ​​are directly matched using regular expressions in memory; for range validation rules (such as personal annual income must be greater than 0), numerical comparisons are performed; for correlation validation rules (such as checking if a customer exists across tables), database queries are executed asynchronously and non-blockingly (CompletableFuture).

[0079] Step 5: Smart Correction and Atomicity Audit like Figure 6 As shown, Figure 6 The detailed flowchart for this step illustrates the complete process of memory correction, change context construction, and nested writes within transactions in automatic silent execution mode.

[0080] When the verification in step 4 determines that the compliance rules are not met, the system enters the correction decision process.

[0081] The core mechanism of the rule engine is as follows: A rule condition network is constructed based on an improved version of the Rete algorithm. The Alpha network filters constant conditions, the Beta network handles variable binding, and shared evaluation results of common conditions are achieved, realizing O(1) constant-level matching when binding 5 to 10 rules to a single field. Nested condition evaluation supports complex Boolean expressions such as IF(AAND(BOR(CANDD)))THEN..., using recursive descent parsing to construct an expression tree, followed by post-order traversal evaluation. Multi-rule conflict handling employs a priority adjudication plus mutex group mechanism: rule definitions use priority (smaller values ​​indicate higher priority), and mutexGroup identifies mutually exclusive rules (only one rule in the same group is executed). In case of conflict, higher priority rules override each other; when rules have the same priority and are in the same group, a RuleConflictException is thrown for manual handling. Execution strategy selection: SUGGEST mode displays suggested corrections in a pop-up window, which is executed after user confirmation; AUTO_SILENT mode executes directly when conditions are met, without user awareness.

[0082] If the current rule is configured for automatic silent execution and the rule's conditional expression is met, the rule engine will directly perform correction operations on the corresponding field values ​​of the business data object in memory. Correction operations include, but are not limited to: RETAIN_FIRST_CHAR (preserve the first character, mask the rest), SM3_HASH (SM3 hash masking), TRIM_AND_UPPER (remove leading and trailing whitespace and convert to uppercase), and DEFAULT_FILL (fill with default values). For example, for an individual customer's name, if the condition "customer type is individual and name length is greater than 1" is met, then "Zhang San San" will be automatically corrected to "Zhang".

[0083] If the current rule is configured with the suggested execution mode, the system will not perform automatic correction but will instead return the correction suggestion to the front end for user confirmation before performing the correction operation. In cases where multiple rules match the same field, the system will decide based on rule priority: rules with lower priority values ​​have higher priority, and the correction result of a higher-priority rule will override the lower-priority rule. If multiple rules have the same priority and belong to the same mutually exclusive group, the system will throw a rule conflict exception, requiring manual handling.

[0084] After memory correction is complete, the system constructs a ChangeContext object. This ChangeContext object contains at least the following: a globally unique change identifier (changeId, UUID format), a business primary key (businessKey, such as an order number used to associate business data), the name of the field being corrected (fieldName), the original value before correction (originalValue, stored in serialized form), the new value after correction (modifiedValue), the triggering rule identifier (ruleId) and rule version number (ruleVersion), the execution mode (executionMode, AUTO_SILENT or SUGGEST), the change type (changeType, SYSTEM_AUTO for automatic system operation or MANUAL_CONFIRM for manual confirmation), the operation timestamp (timestamp, millisecond precision), session information (sessionInfo, including the operation user identifier, client IP address, and session ID), and the server node identifier (serverNode).

[0085] The core challenge in persisting the corrected output lies in ensuring the atomicity of business data modification, change context construction, and audit log persistence without the user's awareness. The solution employs nested writes within a single transaction, with the following steps: 1) The AOP interceptor captures the data object (DTO or Entity) to be persisted, obtains the parameter type through MethodSignature, and identifies the object marked with @Entity.

[0086] 2) The engine pulls binding rules based on metadata tags, performs Rete network matching, and builds an activation rule list (Agenda).

[0087] 3) When the conditions are met, the data object field values ​​are directly modified in memory by calling the Setter method, which supports a variety of transformation functions (RETAIN_FIRST_CHAR, SM3_HASH, TRIM_AND_UPPER, DEFAULT_FILL).

[0088] 4) Construct a ChangeContext object.

[0089] 5) Start a database transaction (using Spring@Transactional declarative transaction, propagating the REQUIRED attribute), which includes two write operations: Operation ① writes the corrected business data to the main business table (JPA / Hibernate's save method, automatic dirty checking), and Operation ② writes the serialized ChangeContext data to an independent audit log table (written after JSON serialization; the audit table and the business table are in the same database but have different schemas, ensuring the availability of local transactions).

[0090] 6) If both operations succeed, the transaction is committed; if either fails (e.g., primary key conflict, serialization exception), the transaction is rolled back. After the transaction is rolled back, the business data remains in its original state, and no audit records are generated, strictly ensuring consistency.

[0091] The key design feature is that ChangeContext is transmitted and persisted along with the business data as an independent entity, rather than being sent asynchronously as a message record afterward. This ensures temporal consistency (the timestamp of the audit record is strictly consistent with the modification time of the business data) and causal relationship (the changeId in the audit record can be used as a foreign key to link to the business data, supporting bidirectional traceability).

[0092] The audit log has the following core innovative features, enabling atomic binding between correction operations and the audit mechanism.

[0093] Append-Only feature: Audit tables are prohibited from UPDATE and DELETE operations, database-level triggers are set to reject modification requests, only INSERT is supported, which meets the "non-repudiation" requirement of financial audits, and any attempt to change data will be recorded.

[0094] WORM storage: The database layer utilizes GaussDB's WORM table feature, setting write_once to true, or uses object storage (such as MinIOWORMBucket) at the underlying level. Audit records are synchronously written twice, providing technical protection against tampering, and even database administrators cannot physically delete them.

[0095] Hash chain structure: Each record contains prevHash (SHA-256 hash of the previous record), currentHash (hash of the current record's content), and chainIndex (chain sequence number). Every day at midnight, the tail record is digitally signed with HMAC-SHA256, with the key held in escrow by a hardware security module (HSM). Tampering can pinpoint the breakpoint, and signature verification ensures chain integrity.

[0096] Logical Isolation and Access Control: Audit log tables and business data tables are deployed in the same database instance, but are logically isolated using different schemas. The system implements strict control at the database user permission level, granting only INSERT permissions to the audit log table to the dedicated audit database account, and revoking its SELECT, UPDATE, and DELETE permissions. Query operations on the audit logs are performed through a separate read-only database copy or a dedicated query interface with temporary read permissions. This ensures transaction atomicity while achieving separation of duties, effectively preventing internal personnel from forging, tampering with, or exceeding their privileges to query sensitive audit data.

[0097] Structured retrieval: Audit records are JSON Schema-based, supporting multi-dimensional index queries by businessKey, fieldName, ruleId, timeRange, etc., to meet the flexible retrieval needs of regulatory inspections.

[0098] An audit log example (JSON format) includes the following fields: recordId (record identifier, e.g., AUD_20260114085500123), chainIndex (chain number, e.g., 1523456), prevHash (hash value of the previous record, e.g., "a3f5c8..."), currentHash (hash value of the current record, e.g., "b7e2d9..."), signature (digital signature, e.g., "MEUCIQDT..."), timestamp (timestamp, e.g., "2026-01-14T08:55:32.123+08:00"), and businessContext (business context, including reportId (report identifier), reportName (report name), businessKey (business primary key), and operationType (operation type, e.g., DATA_...). INSERT, changeDetails (including fieldName, originalValue, modifiedValue, transformationType, ruleTriggered, and ruleVersion), executionContext (including executionMode, operatorId, sessionId, clientIp, and serverNode), and transactionContext (including transactionId and commitTimestamp).

[0099] After the transaction is successfully committed, the system returns a data entry success response to the client.

[0100] Step 6: Output the result set like Figure 7 As shown, Figure 7 The detailed flowchart for this step illustrates the collaborative mechanism between paginated queries and streaming writes.

[0101] When a business user triggers a report export operation, the system retrieves the query result set generated in step 3 or the business data object persisted in step 5.

[0102] The system generates data files using a combination of paginated querying and streaming writes, based on the preset export formats (such as Excel, TXT, and CSV) configured in the report settings. Specifically, the system retrieves data from the database in batches according to a custom pagination value (e.g., 5000 records per page). Each batch of data is processed (e.g., applying data masking rules, formatting date values, and calculating formula values), and then streamed line by line into a workbook object in memory using the SXSSFWorkbook API of the POI library, avoiding loading all data into memory at once. Once all batches of data have been processed, the system writes the workbook object to a temporary file, generates a time-sensitive download link (e.g., valid for 5 minutes), and returns it to the client, who then triggers the file download via this link.

[0103] If the exported data contains fields that have been automatically corrected in step 5, the export process will automatically inherit the corrected values, ensuring that the output data is consistent with the corrected data recorded in the audit log. The data file generator module automatically inherits the row-level and column-level permission filtering results from the report runtime and supports the automatic application of data anonymization rules.

[0104] Example 2 This embodiment provides a metadata-driven report compliance generation system. The system adopts a client-server architecture, and the server can be deployed using a microservice architecture. The modules interact with each other through lightweight communication mechanisms (such as RESTful API or RPC).

[0105] Reference Figure 8 This system comprises, from bottom to top, a heterogeneous data source layer, a metadata mapping module, a configuration management module, an execution engine module, a compliance correction module, a file generation module, and a user terminal layer. Each module works collaboratively through configuration-driven processes, with standardized metadata at its core.

[0106] Figure 8 Explanation of data flow and collaboration relationships: The metadata mapping module collects raw metadata from heterogeneous data source layers and outputs a unified metadata model.

[0107] The configuration management module reads the unified metadata model and generates a semi-structured SQL template.

[0108] The execution engine module receives user requests, retrieves permission policies, injects permission conditions, and then executes the query.

[0109] When data is entered, the compliance correction module sequentially loads rules through the rule base unit, performs verification through the real-time verification unit, performs memory correction through the intelligent correction unit, and performs nested writes within transactions through the atomic audit unit.

[0110] The file generation module obtains the query result set or the corrected business data and completes the streaming export.

[0111] In the user terminal layer, business personnel call the execution engine module and file generation module, while administrators call the metadata mapping module, configuration management module, and compliance correction module for configuration management.

[0112] The specific functions of each part of the system are as follows: Heterogeneous data source layer: The heterogeneous data source layer includes at least one type of database, such as GaussDB, GoldenDB, MySQL, PostgreSQL, and NoSQL databases. These databases store the raw data from the business systems and serve as the target data sources for metadata collection and data querying in this system.

[0113] Metadata mapping module: The metadata mapping module is used to connect to at least one heterogeneous database through a pluggable adapter, collect the raw metadata of each database, and convert the collected raw metadata into a standardized unified metadata model according to a preset unified type mapping rule; each field object in the unified metadata model includes at least a field name, a field type, and a bound compliance rule identifier.

[0114] The specific implementation of this module (including the collection methods for each database type in the adapter framework, unified type mapping rules, primary and foreign key identification methods, automatic supplementation mechanism for business tags and its rule base elements, automated execution process, dynamic update support, etc.) has been described in detail in step 1 of embodiment 1, and will not be repeated here. This embodiment directly references it.

[0115] Configuration Management Module: The configuration management module is used to respond to report configuration operations, obtain the set of fields selected by the user from the unified metadata model and the configured query conditions, and generate a semi-structured SQL template through a declarative compilation pipeline based on the set of fields and the query conditions; the semi-structured SQL template contains a fixed SQL structure and at least one permission injection placeholder.

[0116] The specific implementation of this module (including the four stages of the declarative compilation pipeline—the mapping rules from UI controls collected by the front-end configuration to AST nodes, the semantic verification and predicate pushdown optimization in DSL parsing and IR generation, the field mapping and operator conversion rules in target code generation, and the MD5 hash cache key and LRU eviction policy in the execution plan cache, etc.) has been described in detail in step 2 of embodiment 1, and will not be repeated here. This embodiment directly references it.

[0117] Execution engine module: The execution engine module is used to respond to report query requests, parse the user identity information in the request, construct a security context object; generate permission filtering conditions based on the security context object and the permission policy corresponding to the semi-structured SQL template; inject the permission filtering conditions into the permission injection placeholder of the semi-structured SQL template to generate an executable SQL statement; and execute the executable SQL statement to obtain a query result set.

[0118] The specific implementation of this module (including the data structure definition of the structured policy object, the construction method of the security context, the SpEL expression evaluation mechanism, the recursive generation algorithm of the permission filtering conditions, and the condition injection in the multi-table association scenario, etc.) has been described in detail in step 3 of embodiment 1, and will not be repeated here. This embodiment directly references it.

[0119] Compliance Correction Module: The compliance correction module is the core innovation of this invention, distinguishing it from existing technologies. This module further includes a rule base unit, a real-time verification unit, an intelligent correction unit, and an atomic audit unit.

[0120] The rule base unit is used to store compliance rules. Each compliance rule includes at least a rule identifier, rule type, conditional expression, execution action, and execution mode. The execution modes include automatic silent execution mode and suggested execution mode.

[0121] The real-time verification unit is used to respond to the entry or correction of report data, obtain the business data object to be persisted, load the corresponding compliance rule from the rule base unit according to the compliance rule identifier bound to the field object in the unified metadata model, and verify the field values ​​in the business data object through the rule engine.

[0122] The intelligent correction unit is used to perform correction operations on the corresponding field values ​​in the business data object in memory when the judgment result of the real-time verification unit is not satisfied. If the current compliance rule is configured to be in automatic silent execution mode, the unit will generate a corrected business data object and construct a change context object. The change context object records at least the original value of the corrected field, the corrected value, and the rule identifier that was triggered.

[0123] The atomicity audit unit is used to write the corrected business data object into the business database table and serialize the change context object into the audit log table within the same database transaction, and to return a data entry success response in response to the successful commit of the transaction.

[0124] The specific implementation of this module (including the bidirectional association mode of "metadata anchor point + policy registry", the complete process of rule binding, the dual verification architecture of front-end real-time verification and back-end final fallback, rule matching and nested condition evaluation based on the Rete algorithm, priority adjudication and mutual exclusion group mechanism for multiple rule conflicts, memory correction operation type in automatic silent execution mode, complete data structure of changing context object, six specific steps of nested write within a transaction and its atomicity guarantee, append write feature of audit log, WORM storage mechanism, hash chain structure and digital signature, logical isolation and access control, structured retrieval, etc.) has been described in detail in steps 4 and 5 of embodiment 1, and will not be repeated here. This embodiment directly quotes it.

[0125] File generation module: The file generation module is used to respond to a report export request, obtain the query result set or the corrected business data object, and generate a data file in a streaming manner according to a preset export format and return a download link.

[0126] The specific implementation of this module (including the collaborative mechanism of pagination query and streaming write, the use of the POI library SXSSFWorkbookAPI, temporary file generation and download link return, etc.) has been described in detail in step 6 of embodiment 1, and will not be repeated here. This embodiment directly references it.

[0127] User terminal layer: The user terminal layer includes a business user interface and an administrator interface. The business user interface provides report query, data entry, and data export functions; the administrator interface provides data source configuration, metadata collection triggering, report configuration, compliance rule configuration, permission policy configuration, and audit log query functions.

[0128] System deployment architecture: This system can be deployed using a microservice architecture, with each module deployed independently as a microservice, communicating with each other through a service registration and discovery mechanism. The metadata mapping module, configuration management module, execution engine module, compliance correction module, and file generation module can each be deployed as independent service instances, dynamically scaling up and down based on load. The metadata repository can use a relational database (such as MySQL) or a dedicated metadata service (such as Apache Atlas). The rule and policy repositories can use a tiered storage architecture of a relational database with Redis caching. Audit logs can use a time-series database or a distributed log storage system (such as Elasticsearch) to support efficient multi-dimensional retrieval. The system frontend is implemented using a reactive web framework (such as Vue.js or React), interacting with backend services through a RESTful API.

[0129] Example 3 This embodiment uses the personal customer data report management of a financial institution as a scenario to illustrate the complete implementation process of the technical solution of the present invention in a specific scenario.

[0130] Scenario Description: A financial institution uses GaussDB as its core transaction database and MySQL as its customer information management database. Regulatory authorities require all reports involving personal customer information to anonymize mobile phone numbers, ID card numbers, and customer names, and any data modification operations must be logged in a complete audit log to meet the requirements of the Personal Information Protection Law and financial industry compliance audits.

[0131] Step 1: Metadata Standardization Mapping The financial institution's system administrator logs into the system and adds two data sources in the metadata mapping module's configuration interface: the GaussDB core database (connection parameters: JDBCURL, username, password) and the MySQL customer information database (connection parameters: JDBCURL, username, password). The system connects to the two databases through the GaussDB adapter and the MySQL adapter respectively, automatically collecting metadata.

[0132] During the data collection process, the system identified the following fields in the "customer_info" table of the MySQL customer information database: mobile_phone (VARCHAR type), id_card_no (VARCHAR type), customer_name (VARCHAR type), and annual_income (DECIMAL type). The tag supplementation unit automatically matched according to preset rules in the rule base: the mobile_phone field, because its name contained "phone" and its comment contained "mobile phone number," matched the RULE_PII_PHONE_001 rule, and the tag "Sensitivity Level: PII, De-identification Rule: Mobile Phone Number Mask (Keep the first three and last four digits)" was automatically added; the id_card_no field matched the RULE_PII_IDCARD_001 rule, and the tag "Sensitivity Level: PII, De-identification Rule: ID Card Number Hash (SM3)" was automatically added; the customer_name field matched the RULE_PII_NAME_001 rule, and the tag "Sensitivity Level: PII, De-identification Rule: Name Keeps the First Character" was automatically added. All metadata was integrated into a unified metadata model and stored in the metadata repository.

[0133] Step 2: Dynamic Configuration and SQL Template Generation The administrator creates a report configuration named "Personal Customer Basic Information Details Report". In the visual configuration interface, the administrator selects fields such as legal entity bank, customer number, customer name, ID number, personal annual income, and home address from the unified metadata model, sets the default sorting to ascending order of customer number, and configures the query conditions as "customer number" (text input box, fuzzy matching) and "data date" (date range selector).

[0134] The compilation pipeline unit of the configuration management module converts the above configuration into a semi-structured SQL template: SELECT${ETL_FIELDS}FROMcustomer_infot1WHERE1=1ANDt1.customer_idLIKE?ANDt1.data_dateBETWEEN?AND?${AUTH_FILTER}ORDERBYt1.customer_idASC The template is cached in the template storage unit, and the cache key is the MD5 hash value of the configuration content.

[0135] The administrator further configured compliance rules in the rule base unit of the compliance correction module: bound a range validation rule to the "Individual Annual Income" field, requiring the value to be greater than or equal to 0, with the execution mode set to SUGGEST (suggested mode), and a pop-up prompting correction when the user enters a negative number; bound a transformation correction rule to the "Customer Name" field, automatically de-identifying and retaining the surname when the customer type is "Individual", with the execution mode set to AUTO_SILENT (automatic silent mode); bound a transformation correction rule to the "ID Number" field, automatically performing SM3 hash de-identification when the customer type is "Individual", with the execution mode set to AUTO_SILENT.

[0136] Step 3: Runtime Permission Binding Zhang San, a business employee in the Financial Markets Department, logs into the system to access the "Personal Customer Basic Information Details Report". The request parsing unit of the system execution engine module parses the user identifier "Z001", role "BUSINESS_USER", and institution identifier "800" from Zhang San's token. Based on the report identifier and user role, the permission retrieval unit retrieves the effective permission policy POL_001: restricting the user to only access data from their affiliated institution (branch_id='800') and whose data dates are within the range specified in the request.

[0137] The condition generation unit converts the permission policy into an SQL condition fragment "t1.branch_id=?ANDt1.data_dateBETWEEN?AND?". The query execution unit injects this fragment into the ${AUTH_FILTER} placeholder in the semi-structured SQL template, forming a complete SQL statement, which is then submitted to the database for execution. Zhang San can only query customer data with the organization identifier "800" and cannot access data across organizations.

[0138] Step 4: Data entry and compliance verification Zhang San entered a new customer order on the data entry page attached to the report, filling in the following information: Customer type "Individual", Customer name "Zhang San San", ID number "XXX", and Individual annual income "-100" (which is a negative number due to an input error).

[0139] After filling in the form, Zhang San clicked submit. The front-end real-time validation unit triggered rule R_005 (range validation) when the "Personal Annual Income" field went out of focus. It detected that the value -100 did not meet the "greater than or equal to 0" condition and immediately displayed a pop-up message stating "Personal annual income must be greater than or equal to 0," preventing the form from being submitted. Zhang San corrected his annual income to "150,000" and resubmitted.

[0140] The backend AOP aspect intercepts the submission request, and the real-time validation unit loads the rule engine to perform full validation of the business data objects. The rule engine performs condition matching based on the Rete network: if the customer type is detected as "individual", rules R_003 (customer name anonymization) and R_004 (ID number hash) are activated.

[0141] Step 5: Smart Correction and Atomicity Audit The execution mode of the intelligent correction unit's judgment rules R_003 and R_004 is AUTO_SILENT, and the conditions are met. The correction operation is performed in memory: the customer name "Zhang Sansan" is processed by the RETAIN_FIRST_CHAR transformer, retaining the first character "Zhang" and replacing the remaining characters with an empty string, resulting in "Zhang"; the ID number is processed by the SM3_HASH transformer and converted into a 64-bit hash value. The personal annual income "150000" passes the verification and requires no correction.

[0142] The intelligent correction unit constructs a change context object, which records the original value of two fields before correction, the new value after correction, and the triggering rule identifier.

[0143] The atomic audit unit initiates a database transaction: First, it writes the corrected business data (customer name "Zhang", ID number hash, annual revenue "150000") to the business database table; then, it serializes the change context object into JSON and writes it to an audit log table located in the same database instance as the business table but logically isolated (e.g., in a different schema within the same database). The transaction is successfully committed, and Zhang San completes the automatic data anonymization without any prior knowledge.

[0144] Step 6: Output the result set When Zhang San subsequently exported the report, the customer name in the exported file was displayed as "Zhang," the ID number as a hash value, and the individual's annual income as "150,000." Audit management personnel can log into the audit log query interface to retrieve the complete correction history of this data entry record, including the correction time, a comparison of values ​​before and after the correction, the triggered rule identifier, and the rule version number. By verifying the integrity of the hash chain, it can be confirmed that the audit record has not been tampered with.

[0145] Example 4 This embodiment uses the supply chain reporting system of a large manufacturing enterprise as a scenario to illustrate the implementation process of the present invention in scenarios involving multi-data source association queries and complex access control.

[0146] Scenario Description: A manufacturing company has deployed a GoldenDB distributed database to store order data, and a PostgreSQL database to store supplier information. The company requires that business personnel in different regions can only view order data within their own region, and supplier pricing information is only visible to personnel at specific levels.

[0147] Step 1: Metadata Standardization Mapping The system administrator configures GoldenDB and PostgreSQL data sources. The metadata mapping module aggregates metadata from the order tables across each shard via the GoldenDB dedicated management interface and collects metadata from the supplier table via the pg_catalog system table through the PostgreSQL adapter. The type conversion unit maps GoldenDB's "bigint" and PostgreSQL's "integer" to the standardized type "Numeric," and maps both "varchar" and "text" to "String." The tag addition unit automatically adds the tag "Sensitivity Level: Internal, Access Control Policy: Manager Only" to the "quote_price" field in the supplier table based on the keyword "quote" in the field comments.

[0148] Step 2: Dynamic Configuration and SQL Template Generation The administrator creates an "Order and Supplier Association Report," configuring the order table and supplier table to be joined using the "supplier_id" field in the configuration interface. The compiled pipeline unit generates the following semi-structured SQL template: SELECTt1.order_id,t1.order_amount,t1.region_code,t2.supplier_name,t2.quote_priceFROMorderst1LEFTJOINsupplierst2ONt1.supplier_id=t2.supplier_idWHERE1=1${AUTH_FILTER}ORDERBYt1.order_dateDESC Step 3: Runtime Permission Binding Li Si, a business representative from the North China region, logged into the system to query the report. The execution engine module constructed a security context, with the user role set to "REGION_MANAGER" and the organization identifier set to "NORTH". The permission retrieval unit matched two policies: Policy A (row-level filtering) requires the order region (region_code) to equal the user's region; Policy B (column-level filtering) requires the quote field (quote_price) to be visible only to users with the role "SENIOR_MANAGER", while Li Si's role is "REGION_MANAGER", which does not meet this condition.

[0149] The condition generation unit converts strategy A into the SQL condition fragment "t1.region_code='NORTH'" and injects it into the template's ${AUTH_FILTER} placeholder. The query execution unit executes the SQL, returning all order data for the North China region, but the quote_price field in the result set is filtered to null (NULL). If Li Si attempts to export the report, the file generation module also inherits this column-level permission filtering result, and the quotation information will not appear in the exported file.

[0150] Example 5 This embodiment uses a data reporting scenario of a government system as an example to illustrate the technical safeguards of the present invention in terms of audit traceability and anti-tampering.

[0151] Scenario Description: A municipal government data reporting platform requires each district and county to report statistical data monthly. Data correction operations must meet the audit requirements of "non-repudiation and non-tampering," and audit logs must be kept for more than 10 years.

[0152] Steps 1 to 4 are similar to those in Examples 3 and 4, and will not be repeated here.

[0153] Step 5 Audit Enhancement Implementation: When writing to the audit log, atomic audit units enable a hash chain protection mechanism. The currentHash field of each audit record is calculated as follows: all fields in the audit record except prevHash and currentHash are concatenated into a string, the current timestamp and a random salt value are appended, and the hash value is calculated using the SHA-256 algorithm. The prevHash field is taken from the currentHash value of the previous audit record, and the prevHash of the first record in the chain is set to all zeros. Each audit record also contains a chainIndex field, which increments from 1.

[0154] The atomic audit unit writes audit logs to a separate audit database deployed on a different physical server than the business database. The storage medium is configured in WORM (WriteOnceReadMany) mode, prohibiting modification and deletion of written files at the operating system level. The audit database account only has INSERT privileges; query operations are performed through a dedicated read-only copy, thus separating data writing and querying responsibilities.

[0155] Every day at dawn, the system retrieves a key from the Hardware Security Module (HSM) and performs an HMAC-SHA256 digital signature on the currentHash value of the latest record (with the largest chainIndex) in the audit log table. The signature value is then stored in the signature field of that record. This signature can be used to verify the integrity and authenticity of the audit log afterward—if a record is tampered with, its currentHash value will change, causing all subsequent records' prevHash values ​​to mismatch, and the verification of the tail signature will also fail, thus pinpointing the exact location of the tampering.

[0156] When auditors need to verify the authenticity of a data correction, the system provides a verification tool: it reads the business primary key corresponding to the correction record from the audit log, obtains the complete audit record sequence from the head to the tail, and verifies the integrity of the hash chain of each record and the validity of the tail signature in turn. If the verification passes, it proves that all correction records have not been tampered with since the audit record was generated.

[0157] The scope of protection of this invention is not limited to the specific embodiments described above. Any technical solutions obtained by those skilled in the art through equivalent substitutions, modifications, or variations of the various components, modules, steps, and combinations thereof, inspired by the inventive concept, should fall within the scope of protection of this invention. Specifically, the database types (including but not limited to GaussDB, GoldenDB, MySQL, PostgreSQL), specific transformer types (including but not limited to RETAIN_FIRST_CHAR, SM3_HASH, TRIM_AND_UPPER, DEFAULT_FILL), specific encryption algorithms (including but not limited to SHA-256, HMAC-SHA256, SM3), specific caching mechanisms (including but not limited to Redis, Memcached), and specific persistence frameworks (including but not limited to JPA, Hibernate, MyBatis) involved in this invention are all illustrative examples. Those skilled in the art can choose technical means with similar functions to implement this invention according to the actual deployment environment, and these equivalent substitutions do not depart from the spirit and scope of this invention. Furthermore, the execution order of the method steps described in this invention can be appropriately adjusted without changing the core logic; the division of each module in the system can be merged, split, or renamed according to actual engineering needs. As long as the same functional collaboration is achieved, they should all be regarded as equivalent implementations of this invention.

Claims

1. A metadata-driven method for generating report compliance, characterized in that, include: Step 1: Collect raw metadata from heterogeneous databases through a pluggable adapter, and convert it into a unified metadata model according to a unified type mapping rule. The field objects in the unified metadata model include field names, field types, and bound compliance rule identifiers. Step 2: In response to the report configuration operation, based on the set of fields selected from the unified metadata model and the query conditions, a semi-structured SQL template containing a fixed SQL structure and permission injection placeholders is generated through a declarative compilation pipeline. The declarative compilation pipeline includes a front-end configuration acquisition stage, a DSL parsing and IR generation stage, and a target code generation stage. The front-end configuration acquisition stage converts the UI interface operation into an abstract syntax tree. The DSL parsing and IR generation stage performs semantic verification on the abstract syntax tree and converts it into an intermediate representation. The target code generation stage converts the intermediate representation into a semi-structured SQL template. Step 3: In response to the report query request, construct a security context object, generate permission filtering conditions according to the permission policy, inject the permission injection placeholder, and then execute the SQL statement to obtain the query result set; Step 4: In response to the data entry operation, load the compliance rules according to the compliance rule identifier bound in the unified metadata model, and verify the business data object through the rule engine; Step 5: When the verification fails and the compliance rule is configured in automatic silent execution mode, the system performs a correction operation on the business data object in memory and writes the corrected business data into the business table in the same database transaction. At the same time, the change context recording the correction behavior is written into the audit log table. The automatic silent execution mode means that when the rule conditions are met, the system automatically completes the data correction without the user's awareness and without interrupting the user's operation process. Step 6: In response to the report export request, obtain the query result set or the corrected business data, and generate a data file in a streaming manner.

2. The metadata-driven report compliance generation method according to claim 1, characterized in that, Step 1 also includes automatic supplementation of business tags: establishing a business tag mapping rule library containing trigger conditions, tags to be added, and priority weights; for each field object, extracting the field name, field type, and field comment as matching keys to query the rule library, collecting the matching rules, deduplicating and merging them according to priority, and writing the tags into the field object.

3. The metadata-driven report compliance generation method according to claim 1, characterized in that, The declarative compilation pipeline in step 2 includes: converting UI interface operations into an abstract syntax tree, performing semantic verification on the abstract syntax tree and converting it into an intermediate representation, converting the intermediate representation into a semi-structured SQL template, and calculating a hash value for the normalized intermediate representation as a cache key for caching.

4. The metadata-driven report compliance generation method according to claim 3, characterized in that, The query conditions in the intermediate representation are expressed in a tree structure, supporting arbitrary levels of AND / OR logic nesting.

5. The metadata-driven report compliance generation method according to claim 1, characterized in that, The permission policy in step 3 is stored in a structured policy object containing a rule expression tree and the context of the associated table. When generating permission filtering conditions, a recursive traversal algorithm is used to process the rule expression tree and generate corresponding SQL condition fragments according to the node type.

6. The metadata-driven report compliance generation method according to claim 1, characterized in that, Step 4 adopts a dual validation architecture of front-end real-time validation and back-end final validation: front-end real-time validation performs format, length, required fields, and range validation before user input loses focus or form submission; back-end final validation intercepts methods annotated with compliance checks through AOP, loads the full rule engine, and performs full validation.

7. The metadata-driven report compliance generation method according to claim 1, characterized in that, In step 5, when multiple rules hit the same field, the decision is made based on the priority of the rules, with the rule with the lower priority value overriding the rule with the higher priority value; if multiple rules have the same priority and belong to the same mutually exclusive group, a rule conflict exception is thrown and the case is handled manually.

8. The metadata-driven report compliance generation method according to claim 1, characterized in that, The audit log table adopts an append-only storage strategy, prohibiting update and deletion operations; each audit record contains the hash value of the previous record, the hash value of the current record's content, and the chain number, forming a hash chain structure.

9. A metadata-driven report compliance generation system, characterized in that, The system includes: The metadata mapping module is used to collect raw metadata from heterogeneous databases through a pluggable adapter and convert it into a unified metadata model according to a unified type mapping rule. The field objects in the unified metadata model include field names, field types, and bound compliance rule identifiers. The configuration management module, in response to report configuration operations, generates a semi-structured SQL template containing a fixed SQL structure and permission injection placeholders through a declarative compilation pipeline based on the field set selected from the unified metadata model and query conditions. The declarative compilation pipeline includes a front-end configuration acquisition stage, a DSL parsing and IR generation stage, and a target code generation stage. The front-end configuration acquisition stage converts UI interface operations into an abstract syntax tree. The DSL parsing and IR generation stage performs semantic verification on the abstract syntax tree and converts it into an intermediate representation. The target code generation stage converts the intermediate representation into a semi-structured SQL template. The execution engine module is used to respond to report query requests, construct a security context object, generate permission filtering conditions according to the permission policy, inject the permission injection placeholder, and then execute SQL statements to obtain a query result set. The compliance correction module is used to respond to data entry operations, load compliance rules according to the compliance rule identifier bound in the unified metadata model, and verify the business data object through the rule engine. When the verification fails and the compliance rule is configured in automatic silent execution mode, the correction operation is performed on the business data object in memory, and the corrected business data is written to the business table in the same database transaction. At the same time, the change context recording the correction behavior is written to the audit log table. The automatic silent execution mode means that when the rule conditions are met, the system automatically completes the data correction without the user's awareness and without interrupting the user's operation process. The file generation module is used to respond to report export requests, obtain the query result set or the corrected business data, and generate data files in a streaming manner.

10. A metadata-driven report compliance generation system according to claim 9, characterized in that, The compliance correction module includes: The rule base unit is used to store compliance rules. Each compliance rule includes at least a rule identifier, rule type, conditional expression, execution action, and execution mode. The execution modes include automatic silent execution mode and suggested execution mode. The real-time verification unit is used to load compliance rules based on the compliance rule identifiers bound in the unified metadata model and to verify business data objects through the rule engine. The intelligent correction unit is used to perform correction operations on business data objects in memory and build a change context when the verification fails and the compliance rule is configured to automatic silent execution mode. An atomic audit unit is used to write corrected business data to a business table and serialize the change context before writing it to the audit log table within the same database transaction.

Images