A data-driven intelligent auditing method and system based on a large language model

Abstract

This invention discloses an intelligent data review method and system based on a large language model. It includes: a data quality review step, which calculates statistical indicators of the dataset and inputs them into a first large language model along with sample data to analyze data divergence, distribution anomalies, and outlier issues; an SQL standardization review step, which loads a natural language rule base and inputs it into a second large language model, parses the rules, reviews SQL standardization, and automatically handles rule conflicts; and an SQL duplication review step, which vectorizes SQL statements, retrieves similar historical SQL statements, and inputs them into a third large language model for semantic similarity judgment. The system includes a data quality review module, an SQL standardization review module, an SQL duplication review module, a collaborative scheduling engine, a cache manager, and an asynchronous processing queue. This invention achieves semantic-level duplication detection, automatic natural language rule review, and intelligent quality assessment under limited samples, significantly improving detection accuracy, review efficiency, and the level of intelligent rule management.

Description

Technical Field

[0001] This invention relates to the field of data development and data quality control technology. Specifically, it relates to an intelligent data development auditing method and system based on a large language model, particularly suitable for data quality assessment, SQL compliance auditing, and semantic-level duplicate detection scenarios during the construction of data warehouses in the financial industry. This invention can achieve efficient and accurate data quality early warning and problem diagnosis using limited anonymized sample data during the development phase. Background Technology

[0002] With the deepening of enterprise digital transformation, data warehouse construction has become a core component of enterprise data management. In the financial industry, especially in banking systems, this typically involves thousands of data tables, complex ETL processes, and a large amount of SQL script development. To ensure data accuracy, consistency, and availability, data quality inspection and SQL auditing have become indispensable parts of the data development process.

[0003] In the prior art, patent document CN119149564A proposes a data development review method. This method queries historical SQL statements from tables with the same data source and compares them with the current SQL statement using string similarity (e.g., shortest edit distance). If the similarity exceeds a preset threshold (e.g., 80%), the two are considered similar, and developers are prompted to modify the statements based on the historical SQL. Regarding data quality detection, this method relies on preset statistical test items, such as queries using fixed rules for "total number of rows in the target table," "number of rows after deduplication," and "whether associations are divergent."

[0004] However, the aforementioned existing technologies have the following technical limitations in practical applications: 1. Insufficient accuracy of SQL duplicate detection String similarity-based detection methods struggle to identify SQL statements that are logically identical but syntactically different, such as those with different table aliases, rearranged field orders, varying comment content, or differences between explicit and implicit joins. This approach, which relies solely on surface-level text matching, cannot achieve semantic-level duplicate detection, easily leading to redundant development and workload.

[0005] 2. Data quality inspection has only one dimension. Existing methods rely on fixed statistical rules and can only detect superficial issues such as abnormal row counts and excessively high null value rates. They lack the intelligent analytical capabilities to address complex quality issues such as abnormal data distribution, field divergence, and outliers. Especially in financial data development environments, the development phase typically only allows access to a portion of anonymized sample data, making it impossible to perform full data validation. Traditional quality inspection methods based on full statistical analysis are difficult to implement effectively.

[0006] 3. Low level of automation in SQL specification review Existing SQL specification review technologies mainly have two implementation paths: Path 1: Manual Review Mode Reviewers need to possess both business knowledge (understanding natural language specifications) and technical skills (parsing SQL syntax) to judge whether SQL conforms to the specifications based on their personal experience. This model has the following inherent defects: (1) Review standards vary from person to person, resulting in poor consistency; (2) As the number of specifications increases, the cost of manual memorization and comparison increases exponentially; (3) It cannot cope with high-frequency code submission scenarios.

[0007] Path 2: Rule Engine Review Mode To address the efficiency issues of manual review, the industry has attempted to translate specifications into logic executable by rule engines. However, this translation process itself still relies on manual intervention—specifications written in natural language by business experts (such as "prohibit JOIN operations with severe data skew") must be "translated" by technical personnel into configuration code that rule engines can understand (such as Drools rule files, regular expressions, and script logic). This "natural language → machine language" conversion process is not only a technical bottleneck but also a key step in semantic loss. The difficulty lies in: Specifications involving business context and data distribution characteristics (such as "JOIN to avoid data skew") cannot be fully expressed by simple IF-THEN logic, and complex semantics are difficult to encode. Each update to the specifications requires repeating the entire cycle of "manual understanding → coding → testing → deployment", resulting in high rule maintenance costs. There may be implicit conflicts between multiple rules (such as rule A prohibiting subqueries, rule B recommending subqueries to optimize performance). Traditional rule engines cannot automatically identify these conflicts and rely on manual sorting, making it difficult to handle rule conflicts. Summary of the Invention

[0008] This invention aims to provide a data development intelligent auditing method and system based on a large language model, in order to solve the technical problems in the prior art, such as SQL duplicate detection being based only on strings, incomplete data quality detection, and SQL specification auditing relying on manual or complex rule bases.

[0009] This invention provides a data development intelligent review method based on a large language model, comprising the following steps: Data quality audit steps: Calculate the statistical indicators of the dataset, input the statistical indicators and sample data extracted from the dataset into the first large language model, and let the first large language model analyze whether the dataset has data divergence, distribution anomalies or outliers, and output the problem diagnosis results and optimization suggestions. SQL standardization review steps: Load the rule base stored in natural language form, input the SQL statement to be reviewed and the rules in the rule base into the second language model, the second language model parses the natural language rules and reviews the standardization of the SQL statement to be reviewed, while handling conflicts between rules, and outputs the review results and standardization suggestions; SQL duplication verification steps: Convert the SQL statement to be detected and historical SQL statements into vectors respectively. Retrieve one or more similar historical SQL statements through vector similarity. Input the SQL statement to be detected and the retrieved similar historical SQL statements into the third language model. The third language model determines the semantic similarity between the SQL statement to be detected and the similar historical SQL statements and outputs the duplication detection result.

[0010] This technical solution organically integrates three core functions: data quality audit, SQL compliance audit, and SQL duplication audit, forming a complete intelligent audit loop for data development. By introducing a large language model to achieve semantic-level intelligent analysis, it overcomes the limitations of traditional methods that rely solely on statistical rules or string matching, and solves the technical problems of inaccurate duplication detection, single-dimensional quality detection, and low automation of compliance audit in existing technologies.

[0011] Furthermore, in the data quality review step, the statistical indicators include at least one or more of the following: total number of data rows, field null value rate, number of unique values ​​in a field, and field distribution entropy value; the sample data is anonymized sample data extracted from the dataset; the output of the first large language model includes a confidence field, and the system adopts strategies such as direct adoption, manual review with annotation, or triggering resampling based on the confidence level.

[0012] This technical solution constructs a multi-dimensional statistical indicator system, which can comprehensively evaluate data quality. In view of the practical constraints of having only anonymized sample data in the financial development environment, it uses sample data for evaluation and realizes human-machine collaborative decision-making through a confidence mechanism—high confidence results are automatically adopted, medium confidence results are manually reviewed, and low confidence results trigger resampling, which ensures both automation efficiency and the reliability of key decisions.

[0013] Furthermore, in the SQL standardization review step, the rule base adopts a structured storage format, including a main rule table, a rule dependency table, and a rule version history table. The main rule table at least includes fields for rule identifier, natural language rule description, priority, severity level, version number, effective date, and expiration date. The rule dependency table is used to declare dependency or conflict relationships between rules. The rule version history table is used to record the rule change history. The handling of conflicts between rules includes: based on the rule priority setting mechanism, when a rule conflict is detected, the rules are executed from high to low priority; based on the rule dependency table, mutual exclusion conflicts, overriding conflicts, or concurrent conflicts between rules are automatically identified, and review results are generated according to the preset conflict resolution strategy.

[0014] This technical solution designs a structured rule base storage model, achieving for the first time structured management and automated execution of natural language rules. The main rule table supports metadata management such as priority, version, and effective time; the rule dependency table explicitly declares complex relationships between rules; and the rule version history table supports full lifecycle tracing of rules. Based on this structure, the system can automatically identify and handle rule conflicts (mutual exclusion conflicts, overriding conflicts, and coexistence conflicts), solving the pain points of traditional rule engines where rule conflicts are difficult to identify and rely on manual sorting.

[0015] Furthermore, in the SQL duplication verification step, the vectors are generated using an embedding model optimized for SQL syntax, including Code-BERT, GraphCodeBERT, or a finely tuned Sentence-BERT model; the vector similarity retrieval uses a dynamically adjustable similarity threshold and Top-K parameters, the similarity threshold is set according to statistical distribution, F1-Score optimization, or business trade-offs, and is dynamically adjusted according to SQL type or user feedback; the Top-K parameters are adaptively determined based on the recall curve evaluated from historical data, LLM context window constraints, or SQL complexity.

[0016] This technical solution employs a dedicated embedding model optimized for SQL syntax (such as Code-BERT and GraphCodeBERT), which can capture the abstract syntax tree features and control flow features of SQL statements, generating vectors with stronger semantic representation capabilities, fundamentally improving the robustness of duplicate detection. Simultaneously, it designs scientific threshold setting methods (statistical distribution method, F1-Score optimization method, and business trade-off method) and dynamic adjustment mechanisms (scenario adaptation and feedback-driven adjustment), as well as an adaptive Top-K parameter determination strategy, enabling the system to flexibly optimize retrieval performance according to different scenarios, balancing recall and precision.

[0017] Furthermore, the outputs of the first, second, and third language models are all subject to a multi-level result verification mechanism, which includes one or more combinations of model schema hints, few-shot hints, format verification, confidence assessment, and cross-validation.

[0018] This technical solution employs a multi-layered result verification mechanism to ensure the reliability and consistency of the large language model's output. Model schema hints and few-shot hints guide the model to output according to specifications from the input stage; a format verification layer ensures the output format is valid; a confidence assessment layer adopts differentiated processing strategies based on confidence levels; and a cross-validation layer performs multiple calls to key judgments to obtain voting results. This mechanism solves the technical challenge of uncertainty in the output of large language models, enabling the system to operate stably in real-world production environments.

[0019] This invention also provides a data-driven intelligent auditing system based on a large language model, comprising: The data quality audit module is configured to calculate statistical indicators of the dataset and input the statistical indicators and sample data into the first large language model to obtain the data quality problem diagnosis results and optimization suggestions output by the first large language model. The SQL standardization review module is configured to load a rule base stored in natural language and input the SQL statement to be reviewed and the rules in the rule base into a second language model to obtain the standardization review results and standardization suggestions output by the second language model. The second language model automatically handles conflicts between rules during the review process. The SQL duplication verification module is configured to convert the SQL statement to be detected and historical SQL statements into vectors respectively, retrieve one or more similar historical SQL statements through vector similarity, and input the SQL statement to be detected and the retrieved similar historical SQL statements into a third language model to obtain the semantic similarity judgment result output by the third language model. The collaborative scheduling engine is connected to the data quality audit module, the SQL standardization audit module, and the SQL repetitiveness audit module, respectively, and is used to coordinate the parallel or serial execution of the three modules according to preset strategies or real-time resource conditions. The cache manager is connected to the three modules respectively and is used to cache historical SQL vectors, large language model response results and rule parsing results; An asynchronous processing queue, connected to the cooperative scheduling engine, is used to receive detection tasks and perform asynchronous processing, priority scheduling, and failure retries.

[0020] This technical solution constructs a complete system architecture, organically integrating three core modules with a collaborative scheduling engine, a cache manager, and an asynchronous processing queue. The collaborative scheduling engine supports flexible module orchestration (parallel / serial) to adapt to different resource conditions and business needs; the cache manager significantly improves system performance through triple caching (vector cache, LLM response cache, and rule parsing cache); and the asynchronous processing queue supports concurrent processing, priority scheduling, and reliable retries for large-scale tasks. This architectural design enables the system to efficiently process large-scale SQL detection tasks while ensuring detection accuracy, meeting the dual requirements of real-time performance and throughput in financial data development scenarios.

[0021] Furthermore, the data quality audit module also includes: The indicator calculation unit is used to calculate the statistical indicators, which include at least one or more of the following: total number of data rows, field null value rate, number of unique values ​​in a field, and field distribution entropy value. The sample extraction unit is used to extract desensitized sample data from the dataset as the sample data; The confidence evaluation unit is used to receive the confidence field output by the first large language model, and to adopt strategies such as direct adoption, manual review with annotation, or triggering resampling according to the confidence level.

[0022] This technical solution employs a modular design for the data quality audit module. The indicator calculation unit performs multi-dimensional statistical indicator calculations, the sample extraction unit supports intelligent extraction of anonymized sample data, and the confidence assessment unit implements a human-machine collaborative decision-making mechanism. This modular design improves the system's maintainability and scalability.

[0023] Furthermore, the SQL compliance review module also includes: The rule base management unit is used to store and manage the rule base, which includes a rule master table, a rule dependency table, and a rule version history table. The rule master table contains at least the following fields: rule identifier, natural language rule description, priority, severity level, version number, effective date, and expiration date. The rule dependency table is used to declare the dependency or conflict relationships between rules. The rule version history table is used to record the rule change history. The conflict handling unit is used to automatically identify mutual exclusion conflicts, overriding conflicts, or coexisting conflicts between rules based on rule priority and the rule dependency table, and generate audit results according to the preset conflict resolution strategy.

[0024] This technical solution employs a modular design for the SQL compliance review module, a rule base management unit for structured storage and full lifecycle management of natural language rules, and a conflict resolution unit for automated identification and handling of rule conflicts. This design upgrades rule management from "manual review" to "automated management," significantly reducing rule maintenance costs.

[0025] Furthermore, the SQL duplicate verification module also includes: Vectorization unit for converting SQL statements into vectors using an embedding model optimized for SQL syntax, including Code-BERT, GraphCodeBERT, or a finely tuned Sentence-BERT model; The retrieval unit is used to calculate the similarity between the SQL vector to be detected and the historical SQL vector in the vector database, and to retrieve similar historical SQL statements using a dynamically adjustable similarity threshold and Top-K parameters. The threshold adjustment unit is used to set the initial similarity threshold based on the statistical distribution method, F1-Score optimization method, or business trade-off method, and to dynamically adjust it based on the SQL type or user feedback. The parameter adaptive unit is used to adaptively determine the Top-K parameters based on the recall curve evaluated from historical data, the LLM context window limit, or the SQL complexity.

[0026] This technical solution employs a modular design for the SQL duplicate detection module. The vectorization unit performs semantic vectorization of SQL statements, the retrieval unit enables efficient similarity retrieval, the threshold adjustment unit allows for the scientific setting and dynamic adjustment of similarity thresholds, and the parameter adaptive unit intelligently determines Top-K parameters. These four units work collaboratively to form a complete chain from vectorization to retrieval to parameter optimization, achieving high-precision semantic-level duplicate detection.

[0027] Furthermore, the cache manager includes: Vector cache unit is used to cache the vector representation of historical SQL statements. It adopts the LRU eviction policy and sets expiration time and capacity limit. The LLM response cache unit is used to cache the response results of a large language model. The cache key is generated based on the hash value of the input content and an expiration time is set. The rule parsing cache unit is used to cache the parsing results of the rule base and automatically invalidates the relevant cache when the rule base changes.

[0028] This technical solution employs a modular design for the cache manager. The vector cache unit avoids redundant calculations of historical SQL vectors, the LLM response cache unit reduces the cost overhead of repeated LLM calls, and the rule parsing cache unit ensures real-time consistency of the audit logic after rule changes. This triple caching mechanism optimizes system performance from different dimensions and is a key technology supporting large-scale concurrent detection.

[0029] Compared with the prior art, the present invention has the following significant advantages: 1. Achieving a semantic-level leap in detection accuracy. This invention breaks through the limitations of traditional methods based on string similarity or fixed statistical rules. By combining vectorized retrieval with LLM semantic judgment, it innovatively achieves semantic-level duplicate detection of SQL statements. It can accurately identify SQL statements with identical logic despite superficial differences such as different table aliases, different JOIN syntax, and adjusted field order, fundamentally avoiding redundant development. Simultaneously, by introducing LLM into data quality auditing, it intelligently analyzes data divergence, distribution anomalies, and outliers, achieving detection accuracy far exceeding traditional methods. The LLM output results have a built-in confidence assessment mechanism, supporting the organic integration of manual review and automated decision-making to ensure highly reliable detection results.

[0030] 2. A fundamental improvement in review efficiency. This invention breaks through the technical prejudice that "natural language specifications cannot be directly executed by machines." It stores SQL audit rules directly in natural language form, which are then automatically parsed and executed by a large language model. This completely eliminates the manual conversion step of "business specifications → technical code" in traditional solutions, resolving the semantic gap problem in rule maintenance. The rule base supports priority configuration, version management, and dynamic updates. Rule changes take effect immediately without system downtime, enabling audit strategies to iterate rapidly with business needs. Audit efficiency is several times higher than traditional rule engines.

[0031] 3. Development and maintenance costs are significantly reduced. High-precision semantic duplication detection effectively identifies and avoids redundant development, significantly saving development resources. The Natural Language Rule Base allows business experts to directly write and maintain rules, shifting rule management from the technical team to the business team, lowering the technical threshold and reducing operational costs. The system's built-in LLM-driven optimization suggestion function can automatically locate problems and provide improvement solutions, guiding developers to quickly fix issues and further shortening the development cycle.

[0032] 4. Effectively adapt to data constraints in the development environment To address the practical constraints of only being able to access anonymized sample data during the data development phase, the data quality audit module of this invention utilizes the intelligent analysis capabilities of LLM to effectively assess complex quality issues such as data divergence and distribution anomalies even under limited sample conditions, and provides targeted optimization suggestions (such as adjusting the order of JOIN tables and data skew handling strategies), thus filling the gap in the applicability of traditional full-scale statistical methods in the development environment.

[0033] 5. Achieve intelligent management of complex rule sets. This invention designs a structured rule base storage model that supports rule priority configuration, dependency declaration, version management, and effective time control. Through explicit conflict declaration and multiple conflict resolution strategies (mutual exclusion conflict, overriding conflict, and coexistence conflict), the system can automatically detect and handle complex relationships between rules, solving the pain points of traditional rule engines where rule conflicts are difficult to identify and manually sort out, and achieving efficient, dynamic, and intelligent management of rule sets.

[0034] This invention achieves SQL semantic-level duplicate detection through "vectorized retrieval + LLM semantic judgment", automates the execution of audit rules through "natural language rule base + LLM direct parsing", and realizes intelligent management of rules through structured rule base design. The three work together to form a complete intelligent audit system from code standardization and logical duplication to data quality. It has achieved significant progress over the prior art in terms of accuracy, efficiency, cost and adaptability, fully demonstrating the inventiveness of this invention.

[0035] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the data development and detection method based on a large language model provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the overall system architecture provided in an embodiment of the present invention.

[0037] Figure 3 A detailed flowchart of the data quality audit module provided in this embodiment of the invention.

[0038] Figure 4 A detailed flowchart of the SQL compliance review module provided in this embodiment of the invention.

[0039] Figure 5 A detailed flowchart of the SQL duplicate verification module provided in this embodiment of the invention.

[0040] Figure 6 This is a schematic diagram illustrating the collaborative working mechanism of the three modules provided in an embodiment of the present invention. Detailed Implementation

[0041] It is readily understood that, based on the technical solution of this invention, various embodiments of the invention can be conceived by those skilled in the art without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention. Rather, these embodiments are provided to enable those skilled in the art to gain a more thorough understanding of the invention. Preferred embodiments of the invention are described below in conjunction with the accompanying drawings, which form part of this application and, together with the embodiments of the invention, serve to illustrate the innovative concept of the invention.

[0042] The first aspect of this invention provides a data development detection method based on a large language model, comprising the following steps: Data quality audit steps: Calculate statistical indicators for the dataset, including at least one or more of the following: total number of rows, field null value rate, number of unique values ​​in a field, and field distribution entropy value; input the statistical indicators and sample data extracted from the dataset into the first large language model, which analyzes whether the dataset has data divergence, distribution anomalies, or outlier problems, and outputs problem diagnosis results and targeted optimization suggestions based on the data characteristics of the development environment.

[0043] SQL standardization review steps: Load the rule base stored in natural language format. The rule base adopts a structured storage format and supports rule priority configuration and version management. Input the SQL statement to be reviewed and the rules in the rule base into the second language model. The second language model parses the natural language rules and reviews the standardization of the SQL statement to be reviewed. At the same time, it handles conflicts between rules and outputs the review results and standardization suggestions.

[0044] SQL duplication detection steps: Convert the SQL statement to be detected and historical SQL statements into vectors, which are generated using an embedding model optimized for SQL syntax; retrieve one or more similar historical SQL statements from the historical SQL statement vectors by calculating vector similarity, using a dynamically adjustable similarity threshold and Top-K parameters; input the SQL statement to be detected and the retrieved similar historical SQL statements into a third-largest language model, which determines the semantic similarity between the SQL statement to be detected and the similar historical SQL statements, and outputs the duplication detection result.

[0045] A second aspect of the present invention provides a data development detection system based on a large language model, comprising: The data quality audit module is used to perform the data quality audit steps. The SQL compliance review module is used to perform the SQL compliance review steps. The SQL duplication verification module is used to perform the SQL duplication detection steps. A collaborative scheduling engine is used to coordinate the parallel or serial execution of the data quality audit module, the SQL standardization audit module, and the SQL repetitiveness audit module; The cache manager is used to cache historical SQL vectors, LLM response results, and rule parsing results. The asynchronous processing queue is used for asynchronous processing, priority scheduling, and failure retries of detection tasks.

[0046] Reference Figure 1 and Figure 2 As shown, this invention provides a data development detection method and system based on a large language model. In one specific embodiment, the system can be deployed on a server, which includes a processor and a memory. The memory stores a computer program, and when the processor executes the program, it implements the data development detection method described in this invention.

[0047] To clearly illustrate the technical implementation of this invention, the following description will focus on two dimensions: system architecture and core modules.

[0048] System Architecture: In one specific implementation of the present invention, the system adopts a layered architecture design, consisting of the following four layers: 1. Data Interface Layer This layer is responsible for establishing connections with the target database, executing data sampling SQL and statistical queries to obtain basic metrics. To adapt to the diverse data environments of financial institutions, this layer supports multiple database types, including but not limited to mainstream data storage systems such as MySQL, Oracle, PostgreSQL, and Hive.

[0049] 2. Core Processing Engine The core processing engine encapsulates the business logic of the three core modules of this invention: the data quality audit module, the SQL standardization audit module, and the SQL duplication audit module. An integrated coordinating scheduler within the engine is responsible for coordinating the execution order (serial / parallel) and concurrency control of the three modules based on preset strategies or real-time resource conditions to achieve optimal overall system performance.

[0050] 3. AI Model Service Layer This layer provides inference API interfaces for one or more Large Language Models (LLMs). In actual deployments, either locally deployed open-source models (such as the Qwen series, DeepSeek series, and ChatGLM series) can be used, or cloud service APIs (such as the GPT series) can be called. The AI ​​model service layer is also responsible for managing the frequency control, timeout handling, and exception retry mechanism of model calls to ensure the stability and reliability of the service.

[0051] 4. Data storage layer The data storage layer includes several dedicated storage components: a rule base for storing "natural language rules" (which can be in relational database tables or JSON file format), a vector database (such as Milvus, FAISS, etc.) for storing historical SQL vector indexes, a caching system (such as Redis) for caching intermediate results to improve performance, and a report library for storing detection results.

[0052] Core modules: Based on the above system architecture, in a preferred embodiment of the method of the present invention, three core modules that work in parallel or in cooperation are included: Data quality audit module: responsible for calculating statistical indicators on the input dataset, and combining large language models to perform intelligent analysis of complex issues such as data divergence, distribution anomalies, and outliers, and output quality diagnostic reports and optimization suggestions.

[0053] SQL Standardization Review Module: Responsible for loading the rule base stored in natural language, parsing the rule semantics with a large language model, performing standardization checks on the SQL statements to be reviewed, handling conflicts between rules, and outputting review results and rectification suggestions.

[0054] The SQL duplication verification module is responsible for comparing the SQL statement to be detected with historical SQL statements in a vectorized manner, recalling similar candidates through vector retrieval, and performing semantic-level duplication judgment using a large language model, outputting duplication detection results and analysis explanations.

[0055] The three modules can run independently or be executed in parallel or in sequence through a collaborative scheduler to meet the needs of different application scenarios. Figures 2 to 6 The overall system architecture and detailed workflow of each module are presented. The following sections provide a detailed introduction to each of the three modules.

[0056] Data quality audit module: like Figure 3 As shown, this embodiment provides a data quality auditing method based on a large language model, suitable for intelligent quality assessment of anonymized sample data during the data development phase (especially in the financial industry). The specific implementation steps of this module are as follows: Step 1: Input the original dataset The input data is the raw dataset from the development environment, typically a database table or file. In banking data development scenarios, due to data security and compliance requirements, the input data is anonymized sample data, typically representing 1% to 5% of the total data volume, and sensitive fields have been masked.

[0057] Step 2: Calculation of statistical indicators The system first uses traditional statistical methods to calculate basic data quality indicators. This embodiment defines a complete set of statistical indicators, covering multiple dimensions such as basic statistics, distribution analysis, correlation analysis, and data integrity. Specific indicators, calculation methods, and threshold settings are shown in Table 1.

[0058] Table 1 Calculation methods and threshold settings for statistical indicators Basic Statistics Total number of rows (row_count) COUNT(*) No fixed threshold, used for relative comparison Basic Statistics Field null rate (null_rate) COUNT(field IS NULL OR field = '') / total number of rows × 100% ≤5% (primary key field ≤0.1%) Basic Statistics Count of unique values ​​in a field (distinct_count) COUNT(DISTINCT field) Used to determine the cardinality of a field Distribution Analysis Field distribution entropy <![CDATA[-Σp(x)log2p(x)]]> An entropy value less than 1.0 indicates a skewed distribution. Distribution Analysis Top-N ratio (top_n_ratio) The cumulative percentage of the top N values A top 10 percentage point ratio greater than 90% indicates a high degree of bias. Association analysis Join-cardinality detection Number of rows after JOIN / Number of rows before JOIN Ratio > 1.5 triggers early warning divergence Data integrity Primary key duplication rate (pk_duplicate_rate) (Total number of lines - Number of duplicate lines removed) / Total number of lines × 100% threshold=0% The thresholds mentioned above are primarily based on banking data governance experience. For example, primary key fields cannot be null or duplicated, and the null value rate of key fields must be controlled within a reasonable range. Preferably, different threshold strategies can be configured for different business scenarios. For instance, transaction log tables have a lower tolerance for null values, while log tables allow for a higher null value rate. The system can use a large language model to determine the type of data table and dynamically select the appropriate threshold.

[0059] Step 3: Intelligent Analysis of Large Language Models The system uses the statistical indicators calculated in step 2 and the sample data extracted from the original dataset as input to the first large language model. In this embodiment, the input submitted to the large language model adopts a structured prompt design, which includes complete role settings, task descriptions, input data, and output requirements. The following is an example of a Prompt template for the data quality audit module: [System Character Settings] You are a senior data quality analysis expert with over 10 years of experience in financial data governance.

[0060] You are proficient in data quality assessment methodologies, including multi-dimensional analysis of completeness, accuracy, consistency, and timeliness.

[0061]

Task Description

[0062] (2) Few-Shot Prompting To further improve output stability, this embodiment preferably employs a few-shot prompting technique. When constructing the Prompt, the system automatically injects several standard "data quality issue analysis examples" to guide the model in reasoning according to a preset format and logic. For example, a standard case of "input statistical indicator A → output diagnostic result B" is pre-displayed in the Prompt's "Examples" area. Through this contextual learning mechanism, the large model can quickly align with data quality review standards, reducing the output format error rate to below 5%, significantly better than the zero-shot prompting method.

[0063] (3) Format validation layer The system performs validity checks on the JSON format output by the LLM, including checking the existence of required fields, the correctness of data types, and whether enumerated values ​​are within the allowed range. If the format validation fails, the system will trigger a retry mechanism, retrying a maximum of 3 times.

[0064] (4) Confidence assessment layer The `confidence` field in the LLM output indicates the model's confidence level in the analysis results. The system adopts different strategies based on the confidence level: - Confidence level ≥ 0.8: The analysis results are directly adopted as part of the quality report; - Confidence level 0.5 ~ 0.8: The result is adopted but marked "requires manual review", further verification is recommended; - Confidence level < 0.5: Mark as "uncertain" to trigger more detailed data sampling or manual intervention.

[0065] (5) Cross-validation layer For critical judgments (such as divergence detection), the system can use multiple calls to the LLM and retrieve the voting results to further improve the accuracy of the judgment.

[0066] Step 5: Optimization Suggestions Generation To address the limitation of having only anonymized sample data during the data development phase, this module also includes optimization suggestions based on statistical indicators to help developers improve data quality. Examples of common problems and optimization suggestions are shown in Table 2.

[0067] Table 2 Common Problem Types and Optimization Suggestions** JOIN divergence Data bloat after multi-table joins It is recommended to adjust the JOIN order so that the smaller table drives the larger table; check if there is a one-to-many relationship in the join conditions. Data skew The values ​​of a certain field are highly concentrated. We recommend bucketing the skewed field or using Map Join for optimization; check for the presence of dirty data. Null value exception High null value rate in key fields It is recommended to add data cleaning logic to the upstream ETL process, setting default values ​​or filtering null records. Enumeration value exception Undefined values ​​exist in the enumeration field. It is recommended to establish an enumeration value code table, add data validation rules, and handle outliers uniformly. Step 6: Output a data quality report The module ultimately outputs a data quality report, indicating whether the data passed the test, and provides optimization suggestions based on the analysis of a large language model. Below is an example of a JSON-formatted output: json { "task_id": "DQ-20251115-001", "status": "completed", "result": { "quality_score": 78, "is_divergent": false, "issues": [ { "severity": "medium", "type": "enum_anomaly", "description": "The CUST_LEVEL field was expected to have 5 levels, but 8 values ​​were actually found." "location": "column: CUST_LEVEL", "suggestion": "It is recommended to check the code table definition and handle outliers (99, -1, etc.)." }, { "severity": "low", "type": "null_rate_warning", "description": "CUST_NAME has a 2% null value rate, which is higher than the recommended threshold", "suggestion": "It is recommended to add customer name retrieval logic to the upstream ETL process." } ], "confidence": 0.85, "reasoning": "Based on sample data analysis, the primary key integrity is good, but outliers exist in the enumeration fields, which require attention." } } Through the above steps, this embodiment achieves a multi-dimensional and in-depth evaluation of data quality by combining traditional statistical indicators with intelligent analysis of large language models under limited de-identified sample data conditions, and outputs actionable optimization suggestions, effectively making up for the shortcomings of traditional methods in the development environment.

[0068] SQL compliance review module: When performing SQL standardization auditing, such as Figure 4As shown in the flowchart, the specific implementation steps are as follows: Step 1: Input the SQL to be reviewed and the natural language rule base. The input data consists of two main parts: SQL statements pending review: SQL code submitted by developers that requires compliance checks.

[0069] Natural Language Rule Base: A collection of audit rules stored in natural language, covering multiple dimensions such as code style, performance style, data governance style, and security style.

[0070] In banking data development scenarios, SQL compliance review is a crucial step in ensuring data quality, involving numerous business rules and technical specifications that traditional methods struggle to handle efficiently.

[0071] Step 2: Rule base definition and management This embodiment employs a structured rule base storage model, supporting flexible rule management, efficient rule retrieval, and complex handling of inter-rule relationships. The rule base consists of the following core table structure: (1) Main rule table (sql_audit_rules) rule_id VARCHAR(32) Unique identifier for rules, primary key rule_name VARCHAR(100) Rule Name rule_description TEXT Natural Language Rule Description rule_category VARCHAR(50) Rules can be categorized (e.g., coding standards, performance standards). priority INTEGER Priority (1 highest, 10 lowest), default 5 severity VARCHAR(20) Severity level (e.g., error / warning / info) status VARCHAR(20) Status (active / inactive) version INTEGER Version number, default 1 created_at TIMESTAMP Creation time updated_at TIMESTAMP Update time created_by VARCHAR(50) Creator effective_date DATE Effective Date expiry_date DATE Expiry Date (2) Rule dependency table (rule_dependencies) rule_id VARCHAR(32) Rule ID depends_on_rule_id VARCHAR(32) Dependency rule ID dependency_type VARCHAR(20) Dependency types: require, conflicts with, enhancements primary key (rule_id, depends_on_rule_id) composite primary key (3) Rule version history table (rule_version_history) history_id SERIAL History ID, Primary Key rule_id VARCHAR(32) Rule ID version INTEGER Version number rule_content TEXT Rule content snapshot change_type VARCHAR(20) Change type (created / updated / deleted) change_reason TEXT Reason for change changed_by VARCHAR(50) Change of Person changed_at TIMESTAMP Change time Rule conflict handling mechanism In practical applications, conflicts may arise between rules. This embodiment designs a comprehensive conflict resolution mechanism: (1) Priority setting mechanism: Each rule has a priority attribute (1~10), and the higher priority rule is executed first in case of conflict. Priority division principle: Mandatory regulations (such as safety regulations): Priority 1-3 Performance specifications: Priority 4-6 Code style guidelines: Priority 7-10 (2) Explicit conflict declaration: The conflict relationship between rules is declared through the rule_dependencies table (conflicts_with), and the system automatically identifies and records it when loading rules.

[0072] (3) Conflict resolution strategy: When a rule conflict is detected, the system will automatically handle it according to the preset strategy. The system adopts the following strategy: Mutual exclusion conflict: Only apply high-priority rules, and mark the skipped rules in the report. Coverage conflict: Allow low-priority rules as exceptions, and mark the exceptions in the report. Concurrent conflict: Both rules are executed, and the results are marked separately in the report. The strategies are shown in Table 3.

[0073] Table 3 Rule Conflict Handling Strategies Mutual Exclusion Conflict Rule A prohibits subqueries, while rule B requires the use of subqueries. Execute high-priority rules Execute rule A, skip rule B, and mark the skipped rule in the report. Coverage conflict The general rule prohibits SELECT *, but it is allowed in specific scenarios. Apply exception rules Determine whether an exemption is necessary based on the specific scenario, and indicate any exceptions in the report. Coexisting Conflicts There are slight differences between the two formatting specifications. Both rules are executed, and the results are displayed in a hierarchical manner. The primary standard is marked "Requires Rectification," and the secondary standard is marked "Rectification Recommended." Rule version management mechanism This embodiment supports complete lifecycle management of rules, including rule evolution and rollback: Version number management: Each rule maintains its own version number, which is automatically incremented with each modification. The system retains all historical versions and supports version comparison and quick rollback.

[0074] Effective date control: Rules can be set with effective and expiration dates, and can be enabled and disabled on a scheduled basis to achieve a smooth transition.

[0075] Updated approval process: Rule updates require approval processes including application submission, technical review, business confirmation, and official release to ensure that changes are controllable.

[0076] Canary release mechanism: New rules can be applied to some projects or SQL queries first, and then fully released after the effect is verified, reducing the risk of introduction.

[0077] Rule base example The following is a typical SQL specification example in a bank data development scenario: === Code Standards Class === R001: SELECT * is prohibited; the required list of fields must be explicitly specified. R002: Prohibit the use of functions on indexed columns in the WHERE clause. R003: Complex SQL (more than 50 lines) must include comments explaining the business logic. R004: Hardcoding business parameters in the code is prohibited; variables or configuration tables should be used instead. === Performance Specifications === R005: NOT IN subqueries are prohibited on large tables; NOT EXISTS or LEFT JOIN should be used instead. R006: When performing a multi-table JOIN, the smaller table should be placed on the left side of the JOIN as the driving table. R007: Cartesian product (CROSS JOIN) is prohibited. R008: Partitioned table queries must include the partition field as a filter condition. R009: Avoid using OR in JOIN conditions; instead, split the JOIN into multiple JOINs and then use UNION. === Data Governance Category === R010: Data processing tables must include an ETL_DATE field for date slicing. R011: DELETE and UPDATE operations must include a WHERE condition. R012: TRUNCATE operation is prohibited in production environments. === Safety Regulations === R013: Prohibit the appearance of plaintext sensitive information (ID card, account number, etc.) in SQL statements. Step 3: Large Language Model Analysis and Review The system submits the SQL statement to be reviewed, along with the aforementioned natural language rule base (or a subset thereof), to the second large language model for review. In this embodiment, a carefully designed prompt guides the model to perform the review task. The following is an example of a prompt template: [System Character Settings] You are a rigorous SQL code review robot, specifically responsible for reviewing the SQL compliance in data development scenarios.

[0078] You are familiar with the SQL syntax and best practices of mainstream databases and can accurately identify code style issues.

[0079] [Review Rules Set] Please strictly adhere to all of the following rules during the review process: [Code Style Guide] R001: SELECT * is prohibited; the required list of fields must be explicitly specified. R002: Prohibit the use of functions on indexed columns in the WHERE clause. R003: Complex SQL (more than 50 lines) must include comments explaining the business logic. R004: Hardcoding business parameters in the code is prohibited; variables or configuration tables should be used instead. [Performance Specifications] R005: NOT IN subqueries are prohibited on large tables; NOT EXISTS or LEFT JOIN should be used instead. R006: When performing a multi-table JOIN, the smaller table should be placed on the left side of the JOIN as the driving table. R007: Cartesian product (CROSS JOIN) is prohibited. R008: Partitioned table queries must include the partition field as a filter condition. R009: Avoid using OR in JOIN conditions; instead, split the JOIN into multiple JOINs and then use UNION. [Data Governance] R010: Data processing tables must include an ETL_DATE field for date slicing. R011: DELETE and UPDATE operations must include a WHERE condition. R012: TRUNCATE operation is prohibited in production environments. [Safety Regulations] R013: Prohibit the appearance of plaintext sensitive information (ID card, account number, etc.) in SQL statements. [SQL pending review] SELECT * FROM big_table a JOIN small_table b ON a.id = b.id WHERE DATE_FORMAT(a.created_at, "%Y-%m-%d") = "2024-01-15" Output Requirements Please return the review results in JSON format, including the following fields: - compliant: Whether fully compliant (Boolean value) - Violations: A list of violation details, each violation including rule_id, rule_name, severity, location, description, and suggestion. - overall_score: Overall score (0-100) - confidence: The model's confidence level (0-1) in the audit results. To ensure the reliability and consistency of the LLM output results, this embodiment adopts the same multi-level result verification mechanism as Embodiment 1, including model schema hints, few-sample hints, format verification, confidence assessment, and cross-validation, which will not be elaborated here.

[0080] The verall_score can be derived by the second language model based on a comprehensive evaluation of the number of violations and the severity level, or it can be calculated by the system according to preset scoring rules. For example, the baseline score is 100 points, each error deducts 20 points, each warning deducts 10 points, each info deducts 5 points, and the minimum score is no less than 0 points.

[0081] Step 4: Output SQL audit results The module ultimately outputs structured SQL audit results, including violations, overall score, confidence level, and optimization suggestions. The SQL audit results can leverage the capabilities of large language models to provide normalization suggestions. The output also follows the multi-layered result validation mechanism described in Example 1, which will not be elaborated upon here. The following is an example of the output in JSON format: { "task_id": "SA-20251115-001", "status": "completed", "result": { "compliant": false, "violations": [ { "rule_id": "R001", "rule_name": "Disable SELECT *", "severity": "error", "location": "SELECT clause", "description": "SELECT * was used, but the list of fields was not explicitly specified". "suggestion": "Replace with a specific field name: SELECT a.id, a.name, b.valueFROM..." }, { "rule_id": "R002", "rule_name": "Disallow the use of functions on indexed columns", "severity": "warning", "location": "WHERE clause", "description": "The DATE_FORMAT function was used on the created_at field", "suggestion": "Change to a range query: created_at >= '2024-01-15' AND created_at < '2024-01-16'" }, { "rule_id": "R006", "rule_name": "Small table drives large table", "severity": "info", "location": "JOIN clause", "description": "The large table (big_table) is on the left side of the JOIN, and the small table (small_table) is on the right side." "suggestion": "Adjust the JOIN order: FROM small_table b JOIN big_table a ON ..." } ], "overall_score": 65, "confidence": 0.92 } } Through the above steps, this embodiment realizes the structured management of the natural language rule base and the automated review based on the large language model, effectively solving the technical problems of traditional rule engines such as difficulty in handling natural language rules, difficulty in identifying rule conflicts, and complexity in version management, and significantly improving the efficiency and intelligence level of SQL standardization review.

[0082] SQL duplicate verification module: When performing SQL duplicate checking, such as Figure 5 As shown in the flowchart, the specific implementation steps are as follows: Step 1: Input the historical SQL database and the SQL to be tested Input data includes: Historical SQL statement library: A large number of historical SQL scripts accumulated during the development of the bank's data warehouse, serving as a benchmark for duplicate detection.

[0083] SQL statement to be detected: New SQL code written by the developer that needs to be checked for repetition.

[0084] Step 2: Vectorization Processing The system uses a vectorization model to convert historical SQL statements and SQL statements to be detected into high-dimensional vectors, laying the foundation for subsequent similarity retrieval.

[0085] To improve vectorization performance, the system preprocesses the SQL to be tested and historical SQL statements before vectorization, including removing comments and redundant whitespace, standardizing keyword case, and normalizing string quotation marks. Subsequently, the system uses a vectorization model to convert the preprocessed SQL statements into high-dimensional vectors.

[0086] Vectorization Model Selection: This embodiment preferably uses an embedding model pre-trained for code or structured data, such as Code-BERT, GraphCodeBERT, or Sentence-BERT (S-BERT). The technical reason for choosing such models is that general text embedding models (such as Word2Vec or general BERT) are mainly trained based on the semantic similarity of natural language, tending to classify SQL statements with similar structures but different table names or field names as similar (for example, classifying SELECT name FROM table_A and SELECT name FROM table_B as highly similar, ignoring the differences in business objects); or classifying SQL statements with different structures but the same business logic as dissimilar (for example, explicit JOIN and implicit join). The preferred models in this invention, such as Code-BERT, introduce Abstract Syntax Tree (AST) and code control flow graph features during the pre-training stage, which can effectively capture variable dependencies and data flow features in SQL statements, thereby generating more representative semantic vectors in the vector space, improving the robustness of duplicate detection from a technical perspective.

[0087] Those skilled in the art can also use the company's internal historical SQL data to fine-tune these pre-trained models so that the vector representations they generate are closer to the company's own business terminology.

[0088] Step 3: RAG Search The system employs Retrieval Enhanced Generation (RAG) technology to calculate the similarity (e.g., cosine similarity) between the SQL vector to be detected and all historical SQL vectors in the vector database, and retrieves the Top-K most similar historical SQL statements as candidates.

[0089] Vector database: In this embodiment, a dedicated vector database such as Milvus, FAISS, Pinecone or ChromaDB is preferred to support efficient approximate nearest neighbor (ANN) search.

[0090] Setting and dynamically adjusting the similarity threshold Similarity threshold is a key parameter affecting search performance. This embodiment designs a scientific threshold setting method and a dynamic adjustment mechanism: Initial threshold setting method: (1) Statistical distribution method: Calculate the similarity distribution of the labeled dataset and select the threshold point that best distinguishes positive and negative samples. Usually, the value near the intersection of the positive and negative sample distributions is selected.

[0091] (2) F1-Score optimization method: calculate precision, recall and F1-Score under different thresholds, and select the threshold that maximizes F1-Score.

[0092] (3) Business trade-off method: Adjust the threshold according to business needs. If more attention is paid to recall (finding as many duplicates as possible), the threshold can be lowered; if more attention is paid to accuracy (reducing false alarms), the threshold can be raised.

[0093] The recommended initial threshold range in this embodiment is 0.75 to 0.85. For scenarios requiring high precision, it is recommended to set it to 0.85 or higher.

[0094] Preferably, the system supports a dynamic adjustment mechanism for the similarity threshold to adapt to different scenario requirements: (1) Scenario Adaptation: The threshold is automatically adjusted based on the SQL type (query / update / delete) and complexity. A lower threshold is used for complex SQL to ensure recall, and a higher threshold is used for simple SQL to improve accuracy.

[0095] (2) Feedback-driven adjustment: Collect user feedback on search results (marked as correct / incorrect), and re-evaluate and adjust thresholds periodically based on feedback data.

[0096] Determining Top-K parameters The Top-K parameter determines the number of candidate SQL queries retrieved from the vector library, which has a significant impact on system performance and recall, as shown in Table 4.

[0097] Table 4. Analysis of the Influence of Top-K Parameters K=3 The lower level may have caused similar SQL queries to be missed. The similarity is high; all candidates have high similarity. LLM requires fewer calls and has a faster response time. K=5 Moderate, balancing recall and precision Moderate, introducing a small amount of noise Recommended value, acceptable performance. K=10 Higher, covering more candidates Reduce, including low similarity items LLM calls are frequent and the response is slow. K=20 Highest, covering almost all related aspects Lowest, large number of noise terms Slow response time, not recommended for regular use The Top-K determination method recommended in this embodiment is as follows: (1) Evaluation based on historical data: The recall curves under different K values ​​are evaluated using labeled datasets, and the inflection point where the recall growth tends to flatten is selected as the K value.

[0098] (2) Considering LLM context constraints: The K value setting should ensure that all candidate SQL can be placed in the LLM context window to avoid truncation.

[0099] (3) Adaptive strategy: The system can dynamically adjust the K value according to the complexity of the SQL to be detected. A smaller K value is used for simple SQL and a larger K value is used for complex SQL.

[0100] Step 4: Semantic Judgment of Large Language Model The system takes the SQL statement to be detected and the Top-K similar historical SQL statements retrieved in step 3 as context input to the third language model, which then performs semantic-level repetition judgment. Below is a sample Prompt template for the SQL repetition audit module: [System Character Settings] You are an SQL semantic analysis expert, skilled at determining whether two SQL statements are equivalent or highly similar in business logic.

[0101] You can perform deep semantic analysis while ignoring syntactic differences (such as table aliases, field order, comments, and formatting).

[0102]

Task Description

[0103] Judgment criteria: 1. Completely equivalent: The execution results are exactly the same. 2. Highly similar: The core business logic is the same, with only slight differences in the selection criteria. 3. Partial similarity: Some logical overlap, but with essential differences. 4. Dissimilar: Different business logic [Input Data] SQL to be tested (SQL_A): SELECT t1.cust_name, t2.acct_id FROM CUST_INFO t1 JOIN ACCT_MASTER t2 ON t1.cust_id = t2.cust_id WHERE t1.cust_level = "2" AND t2.acct_status = "A" Historical SQL Candidates (Top-3 Search Results): SQL_B (Similarity: 0.89) SELECT ci.cust_name, am.acct_id FROM CUST_INFO ci, ACCT_MASTER am WHERE ci.cust_id = am.cust_id AND ci.cust_level = "2" AND am.acct_status = "A" Output Requirements Please return the analysis results in JSON format, including the following fields: - duplicate_check_result: Duplicate check results (including has_duplicate, duplicate_type, matched_sql_id, similarity_reason) - analysis_details: Detailed analysis of each candidate SQL (including sql_id, vector_similarity, semantic_similarity, and difference). - confidence: The model's confidence level (0-1) in its overall judgment. To ensure the reliability and consistency of the LLM output results, this embodiment adopts the same multi-level result verification mechanism as Embodiment 1, including model schema hints, few-sample hints, format verification, confidence assessment, and cross-validation, which will not be elaborated here.

[0104] Step 5: Output SQL duplicate detection results The module ultimately outputs structured SQL duplicate detection results, including duplicate judgment, matched historical SQL, analysis details, and confidence level. It also outputs the multi-layered result verification mechanism described in Example 1, which will not be repeated here. The following is an example of the output in JSON format: { "task_id": "RD-20251115-001", "status": "completed", "result": { "duplicate_check_result": { "has_duplicate": true, "duplicate_type": "exact", "matched_sql_id": "SQL_B", "similarity_reason": "SQL_A and SQL_B have completely equivalent business logic; only the table aliases and JOIN syntax differ." }, "analysis_details": [ { "sql_id": "SQL_B", "vector_similarity": 0.89, "semantic_similarity": "high", "difference": "Equivalent conversion: Explicit JOIN vs. implicit join, different table aliases" } ], "confidence": 0.95} } Through the above steps, this embodiment realizes SQL duplication verification by combining vector retrieval with semantic judgment of large language models. It can accurately identify SQL statements with the same logic but different writing styles, effectively avoid redundant development from the semantic level, and significantly improve the accuracy and robustness of detection.

[0105] Reference Figure 6 The overall system design of this invention encompasses the collaborative working mechanism of three core modules (data quality audit module, SQL standardization audit module, and SQL duplication audit module) as well as system performance optimization measures for large-scale detection scenarios. The three modules can operate independently or be flexibly orchestrated through a collaborative scheduler to meet detection tasks under different resource conditions and business requirements.

[0106] Module execution mode Based on system resource availability and business priorities, this embodiment supports flexible configuration of two execution modes: 1. Parallel execution mode When system resources are sufficient (e.g., ample CPU, memory, and LLM call quotas), a parallel execution mode can be adopted to maximize detection efficiency. In this mode, the coordinator simultaneously launches three core modules to perform data quality audit, SQL compliance audit, and SQL duplication audit on the same data development task, respectively. After each module completes its parallel processing, the result aggregator automatically integrates the three detection outputs to generate a comprehensive detection report. Due to parallel execution, the overall execution time is approximately equal to the maximum execution time of each module, improving efficiency by about 40% to 60% compared to serial execution.

[0107] 2. Serial execution mode When system resources are limited or to reduce instantaneous load, a serial execution mode can be configured. In this mode, the three modules are executed sequentially according to a preset order. The recommended default execution order in this embodiment is: SQL standardization review → SQL duplication review → Data quality review. The design logic of this order is as follows: first, the standardization review corrects the format and performance issues of the SQL code itself; then, duplication detection avoids the repeated development of business logic; and finally, data quality is verified on a relatively stable code basis, thus forming a progressive detection closed loop of "code standardization → business logic → data results".

[0108] System performance optimization measures To support large-scale, high-concurrency SQL detection scenarios, this embodiment designs the following performance optimization mechanisms at the system level: 1. Asynchronous processing mechanism Task Queue: After all detection tasks are submitted, they first enter a distributed task queue (such as RabbitMQ or Kafka), where they are asynchronously consumed and processed by a background worker process pool. The task queue supports priority settings (such as high-priority tasks being queued) and batch processing (merging multiple simple tasks), effectively balancing load and response time.

[0109] Callback notification: After the task is completed, the system will proactively notify the caller through a pre-registered callback interface or message queue to avoid the waste of resources caused by the caller continuously polling.

[0110] Progress tracking: The system provides a task progress query interface, allowing callers to obtain the task status (such as waiting, executing, or completed) and estimated remaining time in real time, thus improving the user experience.

[0111] Retry on failure: When a task fails due to a temporary failure (such as network jitter or LLM service timeout), the system automatically triggers a retry mechanism, which can retry up to 3 times, with the interval between each retry increasing exponentially (such as 1s, 2s, 4s) to avoid impacting downstream services.

[0112] 2. Caching mechanism Vector caching: High-dimensional vectors obtained after transforming historical SQL statements using a vectorization model are cached in an in-memory cache (such as Redis). The cache employs an LRU (Least Recently Used) eviction policy and sets reasonable expiration times (such as 24 hours) and capacity limits (such as 100,000 records). When the SQL to be tested is retrieved via vector search, historical SQL vectors are retrieved from the cache first, avoiding redundant calculations and significantly reducing the call pressure on the vectorization model.

[0113] LLM Response Caching: For identical or highly similar input prompts, the system caches the results returned by the large language model. When the cache is hit, the historical result is returned directly, avoiding repeated calls to the LLM inference service, effectively saving costs and improving response speed. The cache key is generated based on the hash value of the input content and has an expiration time (e.g., 1 hour), while also supporting manual clearing.

[0114] Rule parsing caching: The parsing results of the natural language rule base (such as dependencies between rules, conflict matrices, etc.) are also cached. When the rule base changes (such as adding, modifying, or deleting rules), the system automatically invalidates the relevant cache to ensure that the auditing logic is consistent with the latest rules.

[0115] Through the aforementioned collaborative mechanisms and performance optimization measures, this system can efficiently process large-scale SQL detection tasks while ensuring detection accuracy, meeting the dual requirements of real-time performance and throughput in financial data development scenarios.

[0116] Typical application scenarios To illustrate the technical effects of the present invention more intuitively, the following describes two typical application methods of the present invention in conjunction with actual banking application scenarios.

[0117] Scenario 1: Quality Control in Data Warehouse ETL Development During the development of the bank's data warehouse ETL process, after the developers complete the SQL scripts, they conduct a comprehensive test using this system. The specific process is as follows: SQL compliance review: The system automatically loads the coding standards stored internally by the bank in natural language (such as "must include ETL_DATE field" "SELECT * is prohibited", etc.), the large language model parses the rules and reviews the SQL script to be submitted, identifies non-standard parts in the code and provides modification suggestions.

[0118] SQL duplication audit: The system compares newly developed SQL scripts with the historical ETL script library using vectorization. Through RAG retrieval and LLM semantic judgment, it identifies whether there are existing scripts with logical duplication, effectively avoiding redundant development.

[0119] Data quality audit: In the development environment, the system calculates statistical indicators based on desensitized sample data, and uses a large language model to intelligently analyze issues such as data divergence, distribution anomalies, and outliers, thus identifying potential data quality problems in advance.

[0120] Application Results: After adopting this system, a bank saw a 35% improvement in ETL code standardization, a 28% reduction in redundant development, and a 45% increase in the early detection rate of data quality issues, significantly reducing development and maintenance costs.

[0121] Scenario 2: Data Governance Platform Integration This system can be integrated as a core component into the bank's data governance platform, serving as a mandatory quality checkpoint in the data development process. CI / CD Integration: Integrates the system into the continuous integration / continuous deployment pipeline, automatically triggering checks during code commits. For code that fails SQL compliance review or duplication checks, the system can block merge operations, ensuring that only code that meets quality standards enters the codebase.

[0122] Regular inspections: The system can be configured with scheduled tasks to perform periodic quality checks on critical SQL statements in the production environment (such as daily batch scripts), promptly identify quality issues caused by data changes (such as newly added outliers, increased JOIN divergence, etc.), and issue warnings.

[0123] Knowledge Accumulation: The system automatically summarizes detection results and optimization suggestions, forming a searchable knowledge base. Developers can use the knowledge base to learn about common problems and best practices, continuously improving their development skills; governance personnel can use the knowledge base to analyze quality trends and optimize governance strategies.

[0124] It should be noted that the structure and data items of the prompts in the above embodiments are merely illustrative examples. In actual operation, they can be flexibly adjusted according to business needs and data characteristics, and their specific content does not constitute a limitation on the present invention.

[0125] It should be noted that, in this embodiment, the "first large language model," "second large language model," and "third large language model" are logically entities that perform different review tasks. However, in physical implementation, they can be the same deployed instance of a large language model, calling and executing their respective functions through different prompt projects. This design ensures both logical clarity of functions and reduces the complexity of system deployment.

[0126] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this invention should be included within the protection scope of this invention.

[0127] It should be understood that, in order to simplify the present invention and help those skilled in the art understand its various aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes described in a single embodiment or with reference to a single figure. However, the present invention should not be construed as including all features in the exemplary embodiments as essential technical features of the claims of this patent.

[0128] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0129] It should be understood that the modules, units, components, etc., included in the device of one embodiment of the present invention can be adaptively changed to be placed in a device different from that embodiment. Different modules, units, or components included in the device of the embodiment can be combined into a single module, unit, or component, or they can be divided into multiple sub-modules, sub-units, or sub-components.

[0130] The modules, units, or components in the embodiments of the present invention can be implemented in hardware, in software running on one or more processors, or in a combination thereof. Those skilled in the art should understand that... In practice, microprocessors or digital signal processors (DSPs) can be used to implement embodiments of the invention. The invention can also be implemented on computer program products or computer-readable media for performing some or all of the methods described herein.

Claims

1. A data-driven intelligent review method based on a large language model, characterized in that, Includes the following steps: Data quality audit steps: Calculate the statistical indicators of the dataset, input the statistical indicators and sample data extracted from the dataset into the first large language model, and let the first large language model analyze whether the dataset has data divergence, distribution anomalies or outliers, and output the problem diagnosis results and optimization suggestions. SQL standardization review steps: Load the rule base stored in natural language form, input the SQL statement to be reviewed and the rules in the rule base into the second language model, the second language model parses the natural language rules and reviews the standardization of the SQL statement to be reviewed, while handling conflicts between rules, and outputs the review results and standardization suggestions; SQL duplication verification steps: Convert the SQL statement to be verified and the historical SQL statements into vectors respectively. Retrieve one or more similar historical SQL statements through vector similarity. Input the SQL statement to be verified and the retrieved similar historical SQL statements into the third language model. The third language model determines the semantic similarity between the SQL statement to be verified and the similar historical SQL statements and outputs the duplication detection result.

2. The method according to claim 1, characterized in that, In the data quality review step, the statistical indicators include at least one or more of the following: total number of data rows, field null value rate, number of unique values ​​in a field, and field distribution entropy value; the sample data is anonymized sample data extracted from the dataset; the output of the first large language model includes a confidence field, and the system adopts a strategy of direct adoption, manual review with annotation, or triggering resampling based on the confidence level.

3. The method according to claim 1, characterized in that, In the SQL standardization review step, the rule base adopts a structured storage format, including a rule master table, a rule dependency table, and a rule version history table; the rule master table at least contains fields for rule identifier, natural language rule description, priority, severity level, version number, effective date, and expiration date; the rule dependency table is used to declare the dependency or conflict relationships between rules; The rule version history table is used to record the rule change history; The conflict between the processing rules includes: based on the rule priority setting mechanism, when a rule conflict is detected, the rules are executed in descending order of priority; Based on the rule dependency table, the system automatically identifies mutual exclusion conflicts, overriding conflicts, or coexistence conflicts between rules and generates audit results according to the preset conflict resolution strategy.

4. The method according to claim 1, characterized in that, In the SQL duplication review step, the vectors are generated using an embedding model optimized for SQL syntax; the vector similarity retrieval uses a dynamically adjustable similarity threshold and Top-K parameters. The similarity threshold is set according to statistical distribution, F1-Score optimization, or business trade-offs, and is dynamically adjusted based on SQL type or user feedback; the Top-K parameters are adaptively determined based on recall curves evaluated from historical data, LLM context window limits, or SQL complexity.

5. The method according to claim 1, characterized in that, The outputs of the first, second, and third language models all undergo a multi-level result verification mechanism, which includes one or more combinations of model schema hints, few-sample hints, format verification, confidence assessment, and cross-validation.

6. A data-driven intelligent review system based on a large language model, characterized in that, include: The data quality audit module is configured to calculate statistical indicators of the dataset and input the statistical indicators and sample data into the first large language model to obtain the data quality problem diagnosis results and optimization suggestions output by the first large language model. The SQL standardization review module is configured to load a rule base stored in natural language and input the SQL statement to be reviewed and the rules in the rule base into a second language model to obtain the standardization review results and standardization suggestions output by the second language model. The second language model automatically handles conflicts between rules during the review process. The SQL duplication verification module is configured to convert the SQL statement to be detected and historical SQL statements into vectors respectively, retrieve one or more similar historical SQL statements through vector similarity, and input the SQL statement to be detected and the retrieved similar historical SQL statements into a third language model to obtain the semantic similarity judgment result output by the third language model. The collaborative scheduling engine is connected to the data quality audit module, the SQL standardization audit module, and the SQL repetitiveness audit module, respectively, and is used to coordinate the parallel or serial execution of the three modules according to preset strategies or real-time resource conditions. The cache manager is connected to the three modules respectively and is used to cache historical SQL vectors, large language model response results and rule parsing results; An asynchronous processing queue, connected to the cooperative scheduling engine, is used to receive detection tasks and perform asynchronous processing, priority scheduling, and failure retries.

7. The system according to claim 6, characterized in that, The data quality audit module also includes: The indicator calculation unit is used to calculate the statistical indicators, which include at least one or more of the following: total number of data rows, field null value rate, number of unique values ​​in a field, and field distribution entropy value. The sample extraction unit is used to extract desensitized sample data from the dataset as the sample data; The confidence evaluation unit is used to receive the confidence field output by the first large language model, and to adopt strategies such as direct adoption, manual review with annotation, or triggering resampling according to the confidence level.

8. The system according to claim 6, characterized in that, The SQL standardization review module also includes: The rule base management unit is used to store and manage the rule base, which includes a rule master table, a rule dependency table, and a rule version history table. The rule master table contains at least the following fields: rule identifier, natural language rule description, priority, severity level, version number, effective date, and expiration date. The rule dependency table is used to declare the dependency or conflict relationships between rules. The rule version history table is used to record the rule change history. The conflict handling unit is used to automatically identify mutual exclusion conflicts, overriding conflicts, or coexisting conflicts between rules based on rule priority and the rule dependency table, and generate audit results according to the preset conflict resolution strategy.

9. The system according to claim 6, characterized in that, The SQL duplicate verification module also includes: Vectorization units are used to convert SQL statements into vectors using an embedding model optimized for SQL syntax. The retrieval unit is used to calculate the similarity between the SQL vector to be detected and the historical SQL vector in the vector database, and to retrieve similar historical SQL statements using a dynamically adjustable similarity threshold and Top-K parameters. The threshold adjustment unit is used to set the initial similarity threshold based on the statistical distribution method, F1-Score optimization method, or business trade-off method, and to dynamically adjust it based on the SQL type or user feedback. The parameter adaptive unit is used to adaptively determine the Top-K parameters based on the recall curve evaluated from historical data, the LLM context window limit, or the SQL complexity.

10. The system according to claim 6, characterized in that, The cache manager includes: Vector cache unit is used to cache the vector representation of historical SQL statements. It adopts the LRU eviction policy and sets expiration time and capacity limit. The LLM response cache unit is used to cache the response results of a large language model. The cache key is generated based on the hash value of the input content and an expiration time is set. The rule parsing cache unit is used to cache the parsing results of the rule base and automatically invalidates the relevant cache when the rule base changes.

Images