A natural language data cockpit end-to-end generation method and system

Abstract

This application discloses an end-to-end generation method and system for a natural language data dashboard. The method includes: receiving a natural language requirement input by a user; parsing the natural language requirement and converting it into a structured query intent; generating at least one database query statement and a corresponding visualization specification description based on the structured query intent and the pattern information of the target database, and forming a query mapping object; starting a multi-source SQL validator to verify the at least one database query statement; after the database query statement is verified, generating a front-end page template according to the visualization specification description, encrypting the database query statement, and injecting the encrypted query statement into the front-end page template to generate a preview page; performing end-to-end verification on the generated preview page; and performing offline deployment processing after successful verification. This application can realize a process of converting natural language requirements into a data visualization page that meets comprehensive requirements.

Description

Technical Field

[0001] This application relates to the field of data processing and visualization technology, specifically to an end-to-end generation method and system for a natural language data cockpit. Background Technology

[0002] In the interdisciplinary field of data visualization and natural language processing, with the continuous development of enterprise-level business systems, the demand for scenarios such as data dashboards, single-chart analysis pages, and thematic data dashboards is increasing. These visualization pages help enterprises present data more intuitively, assist in decision-making, and improve business operational efficiency. Traditional data visualization technologies mainly focus on displaying data from databases through charts and reports. However, in recent years, the integration of natural language processing technology has enabled users to express their data needs in a more natural way. Nevertheless, how to achieve efficient, secure, and accurate conversion of natural language requirements into deployable data visualization pages remains a significant challenge in this field.

[0003] To address the challenge of translating natural language requirements into data visualization pages, existing technologies employ various methods. Traditional BI / data visualization platforms, such as Tableau, Power BI, FineReport BI, and QuickBI, typically require users to manually configure charts and data sources through drag-and-drop visualizations, or require analysts with SQL writing skills to directly write SQL and bind it to chart components. Natural Language to SQL (NL2SQL) models proposed by academia and industry, such as DIN-SQL, CodeS, and Spider dataset-driven deep learning models, attempt to solve the "natural language to SQL" conversion problem. Some companies also use a solution of "front-end developers pre-writing HTML pages + embedding SQL + back-end template rendering" to generate static HTML data pages.

[0004] However, existing technologies have significant shortcomings, making it difficult to simultaneously meet the comprehensive requirements of ease of use for business users, secure SQL transmission, cross-database dialect adaptation, rapid generation of visual pages, and operational stability. Traditional BI platforms have high barriers to entry, requiring users to have database SQL writing skills and BI tool operation skills, making them difficult for business users to use directly. Furthermore, the process from requirement submission to page generation requires multi-party collaboration, resulting in a long cycle, and issues such as data sensitivity and dialect fragmentation also exist. The NL2SQL model only generates SQL, not visualizations, lacks intent alignment mechanisms, does not support complex dependency orchestration, and has limited cross-dialect capabilities. Static HTML data pages have high modification costs, expose SQL in plaintext, lack validation mechanisms, rely on external CDNs, and pose stability and availability risks. Summary of the Invention

[0005] To achieve a comprehensive process of converting natural language requirements into data visualization pages, this application provides an end-to-end method and system for generating natural language data dashboards.

[0006] Firstly, this application provides an end-to-end method for generating a natural language data cockpit, including: S1. Receive the user's input natural language request, parse the natural language request and convert it into a structured query intent; the structured query intent includes at least the data range and binding information of the chart to be generated; S2. Based on the structured query intent and the schema information of the target database, generate at least one database query statement and its corresponding visual specification description, and form a query mapping object; the query mapping object is associated with at least the database query statement, the visual specification description, and data dependencies; start a multi-source SQL validator to verify the at least one database query statement; S3. After the at least one database query statement is verified, a front-end page template is generated according to the visualization specification description. The at least one database query statement is encrypted and the encrypted query statement is injected into the front-end page template to generate a preview page. S4. Perform end-to-end verification on the generated preview page; after successful verification, perform offline deployment processing; the offline deployment processing includes: downloading the external network resources referenced in the generated preview page to the local machine, updating the reference links to the external network resources in the page, and generating a data visualization page that supports independent operation.

[0007] By adopting the above solution, business personnel can generate data dashboards using natural language without needing to master technical details such as SQL, HTML, and ECharts, achieving efficient generation from requirement submission to page deployment; setting up multi-source SQL validators to intercept SQL errors during the generation phase eliminates the need for developers to debug at runtime; encrypting database query statements prevents business SQL from being exposed in plaintext on the front end; setting up offline deployment ensures that the generated pages no longer rely on external CDNs at runtime, improving stability and availability; the entire process follows a four-stage pipeline mechanism, allowing each stage's output to be independently inspected and traced back, ensuring the quality of the generated data dashboard.

[0008] Preferably, step S1 includes: S1.1 Explicitly convert the received natural language requirements into a structured intent object; the structured intent object includes at least: a data range field for defining the entire page data range, a chart binding array for describing the binding relationship between each chart and the data source, an ambiguity array for recording the ambiguities that need to be resolved, and a semantic confirmation array for recording the semantic items that need to be confirmed. S1.2. Based on the pattern information of the target database and the predefined business rules, actively resolve ambiguities of the structured intent object; for the identified potential ambiguities, first query the knowledge base for inference, and only add the corresponding item to the ambiguity array when the knowledge base cannot provide a clear answer; S1.3. Through at least one round of interactive dialogue with the user, resolve the ambiguity of items in the ambiguity array and confirm or correct items in the semantic confirmation array. S1.4 After all items in the ambiguity array have been resolved and all items in the semantic confirmation array have been confirmed or corrected, a structured query intent containing a clear data range and binding information is generated based on the structured intent object.

[0009] By adopting the above approach, through explicit mechanisms, proactive ambiguity resolution, and user interaction confirmation, the originally vague natural language requirements are transformed into structured and accurate unambiguous query intents, effectively eliminating the gap between the ambiguity of natural language and the rigor of SQL.

[0010] Preferably, step S2 includes: S2.1. Based on the chart binding information in the structured query intent, generate a query mapping file, and each query mapping in the query mapping file includes at least: a database query statement, a visualization specification description, and data dependencies; wherein, the visualization specification description is used to define the chart type and numerical fields; the data dependencies are used to declare the data source and parameter source information required to execute the query; S2.2. Start the multi-source SQL validator to verify the database query statements in the query mapping file; the verification process includes: dual-source execution verification, incremental retest verification, error handling and prompting, and incremental result comparison; the dual-source execution verification verifies the correctness of the same database query statement through at least two different execution channels; the execution channels include: a first channel directly connected to the target database, and a second channel through calling the backend application interface; the incremental retest verification is performed through multiple rounds of verification, with each subsequent verification round only targeting database query statements that failed in the previous verification round; the error handling and prompting automatically retryes database query statements that fail due to network connection abnormalities; and generates corresponding correction prompts based on the type of verification failure; the incremental result comparison compares the current verification result with the historical verification result and marks the database query statements whose status has changed.

[0011] By adopting the above approach, a query mapping file is generated, clearly defining the chart type and numerical fields, and specifying the data source and parameter source information required for the query. A multi-source SQL validator is set up to validate the database query statement, thereby effectively intercepting SQL errors during the generation phase and avoiding debugging by developers at runtime.

[0012] Preferably, step S2 further includes: S2.0, Establish an extensible database dialect abstraction layer; the database dialect abstraction layer defines a unified syntax conversion interface; In step S2.1, when generating the query mapping file, an intermediate query description independent of the database dialect is generated based on the structured query intent and the pattern information of the target database. Step S2.1 further includes: selecting the corresponding dialect implementation from the database dialect abstraction layer according to the type of the target database; using the selected dialect implementation, converting the intermediate query description into a database query statement that conforms to the syntax of the target database, and filling it into the query mapping file; Step S2.2 also includes: when the multi-source SQL validator performs validation, it calls the database dialect abstraction layer to perform syntax compliance preprocessing on the database query statement related to the target database.

[0013] By adopting the above solution, the core code is decoupled from the specific database, and a single codebase can support multiple databases. This solves the problem of needing to maintain a separate set of SQL templates and validation logic for each database dialect, reducing development and maintenance costs and enhancing code scalability and cross-database compatibility.

[0014] Preferably, the implementation of selecting the corresponding dialect from the database dialect abstraction layer in step S2.1 includes intelligent conversion logic for processing the database's preset grammar rules; The intelligent conversion logic includes: for the Oracle database dialect implementation, its pagination processing logic is implemented by constructing multi-level nested subqueries, and includes an idempotency check mechanism to ensure that multiple pagination operations will not cause syntax errors; for the DM database dialect implementation, its reserved word processing logic adopts a hierarchical strategy: the first-level reserved word set is used to explicitly escape identifiers with quotation marks, and the second-level reserved word set is used to perform column name-level pattern matching and replacement in the entire SQL string, and avoids escaping SQL syntax keywords.

[0015] By adopting the above solutions, the adaptation problems caused by the specific syntax rules of different databases are solved. For Oracle databases, the pagination processing logic of multi-level nested subqueries, combined with the idempotency verification mechanism, avoids syntax errors caused by multiple pagination operations. For DM databases, the hierarchical reserved word processing strategy can accurately handle column name-level reserved words and SQL syntax keywords, ensuring that the database query statement conforms to the syntax requirements of the target database.

[0016] Preferably, step S3 includes: S3.1. Generate the front-end page template based on a predefined, minimized code skeleton containing core code interaction and data acquisition logic; S3.2. Encrypt the at least one database query statement and inject the encrypted query statement into a specified position in the front-end page template; S3.3 Perform multi-level verification on the front-end page template after injecting encrypted query statements; the multi-level verification includes: placeholder integrity verification, API contract verification, and code security verification; the placeholder integrity verification refers to checking whether the front-end page template contains preset key data and configuration placeholders; the API contract verification refers to verifying whether the data request logic in the front-end page template conforms to the predefined back-end application interface specification; the code security verification refers to detecting whether there are preset categories of coding defects or security risks in the front-end page template through static analysis; S3.4 After the verification is passed, a preview page is generated based on the front-end page template and the injected encrypted query statement.

[0017] By adopting the above scheme, the efficiency and accuracy of template generation are improved by generating front-end page templates based on the minimized code skeleton, the encryption of database query statements can prevent the leakage of business logic, and the multi-level verification ensures that the front-end page template contains necessary placeholders, conforms to API specifications and has no security risks, and generates high-quality preview pages.

[0018] Preferably, the front-end page template in step S3.1 includes: a data loading execution function; the data loading execution function is generated based on all data dependencies in the query mapping file, and the execution logic of the function is arranged to follow preset dependency constraints; the preset dependency constraints include: order constraints, parameter binding constraints, and execution optimization constraints; the order constraint means that for queries with dependencies, the execution of the dependent query is completed before the execution of the dependent query begins; the parameter binding constraint means that the parameters required when the dependent query is executed must be extracted and bound from the result data returned by the query it depends on, according to the data source, row index, and field name specified in the traceability information list; the execution optimization constraint means that queries without direct or indirect dependencies are grouped to achieve parallel execution within the group and sequential execution between groups; The static analysis in step S3.3 includes scanning the data loading execution function to identify and block execution logic that violates the preset dependency constraints.

[0019] By adopting the above approach, we can ensure that the execution order, parameter binding, and execution optimization of queries in multi-chart pages comply with preset dependency constraints, avoid errors caused by execution logic not conforming to constraints, improve the accuracy and efficiency of data loading, and identify and block illegal execution logic during the generation stage, thereby reducing the cost of later debugging.

[0020] Preferably, the encryption process in step S3.2 includes: Generate a random initialization vector; use the pre-shared encryption key and initialization vector to perform symmetric encryption on the database query statement to generate ciphertext; based on the pre-shared authentication key, calculate the combination of initialization vector and ciphertext to generate a message authentication code; concatenate and encode the initialization vector, ciphertext, and message authentication code to generate the encrypted query statement string.

[0021] By adopting the above scheme, we can effectively prevent the plaintext of business SQL from being exposed on the front-end page, avoid the leakage of business logic, and at the same time use message authentication codes to verify the ciphertext, promptly identify ciphertext tampering, resist time-series side-channel attacks, and ensure the security of SQL transmission.

[0022] Preferably, step S4 includes: S4.1 Perform end-to-end verification on the generated preview page; the end-to-end verification steps include: when the front-end page is running, submit the encrypted query statement string and its corresponding initialization vector to the back-end application interface, so that after the back-end receives the data, it performs security decryption and query operations, verifies the correctness of the message authentication code, and rejects the request if the verification fails; after the verification is successful, use the encryption key and initialization vector to decrypt the ciphertext and restore the database query statement; execute the restored database query statement using parameterized query method; S4.2 After successful verification, perform offline deployment processing; the offline deployment processing includes: scanning the preview page, identifying and extracting all external network resource addresses referenced therein; for the extracted external network resource addresses, performing a download operation and saving them to a specified local resource directory; wherein, for cascading style sheet files, after downloading, further scanning their content to recursively download the associated resources they reference; replacing the references to external network resource addresses in the preview page with relative paths pointing to the corresponding files in the local resource directory; synchronizing the updated page files and the local resource directory to the target deployment location, generating a data visualization page that supports independent operation.

[0023] By adopting the above scheme, end-to-end verification can ensure the security and correctness of encrypted query statements, prevent ciphertext from being tampered with, and avoid insecure data requests; setting up offline deployment processing means that the generated pages no longer depend on external networks when running, improving the stability and availability of the pages, and making it easier for business personnel to use the generated, independently running data visualization pages.

[0024] Secondly, this application provides an end-to-end natural language data cockpit generation system, including: The intent alignment module is used to receive the user's input natural language request, parse the natural language request and convert it into a structured query intent; the structured query intent includes at least the data range and binding information of the chart to be generated. The SQL generation module is used to generate at least one database query statement and a corresponding visual specification description based on the structured query intent and the schema information of the target database, and form a query mapping object; the query mapping object is associated with at least the database query statement, the visual specification description and data dependencies; and a multi-source SQL validator is started to verify the at least one database query statement. The template generation module is used to generate a front-end page template according to the visual specification description after the at least one database query statement is verified, to encrypt the at least one database query statement, and to inject the encrypted query statement into the front-end page template to generate a preview page. The page arrangement module is used to perform end-to-end verification of the generated preview page; after successful verification, offline deployment processing is performed; the offline deployment processing includes: downloading the external network resources referenced in the generated preview page to the local machine, updating the reference links to the external network resources in the page, and generating a data visualization page that supports independent operation.

[0025] By adopting the above solution, business personnel can generate a data dashboard with zero code, shortening the time from requirement submission to page deployment. Business SQL is encrypted during transmission to prevent leakage, ensuring zero errors in dependency orchestration, early detection of SQL errors, enabling the generated page to run offline, and ensuring that the entire generation process is auditable and quality controllable.

[0026] In summary, this application has the following beneficial effects: 1. Receive natural language input from users, parse and transform it into structured query intent, then generate database query statements, visual specification descriptions and query mapping objects, then inject them into the front-end page template through encryption to generate a preview page, and finally verify and deploy offline, realizing end-to-end generation from natural language to data visualization pages, meeting the comprehensive requirements of ease of use for business personnel and rapid generation of visualization pages; 2. Transform natural language requirements into structured query intents, proactively resolve ambiguities, interact with users to confirm semantics, and eliminate the differences between natural language and SQL; establish a database dialect abstraction layer, generate intermediate query descriptions and convert them into target syntax, combine multi-source SQL verification to achieve cross-database adaptation and intercept errors in advance; encrypt and decrypt database query statements during transmission to prevent exposure of business SQL, and promptly identify tampering to meet secure transmission requirements; declare data dependencies, automatically load them in topological order, and use a rule engine to block illegal logic to avoid dependency orchestration errors; perform end-to-end verification of preview pages, deploy and download external resources offline, update references, and improve page stability. Attached Figure Description

[0027] Figure 1 This is a flowchart of the end-to-end natural language data cockpit generation method described in a specific embodiment; Figure 2 This is a flowchart illustrating the four-stage rigorous operation of the end-to-end natural language data cockpit generation method described in a specific embodiment; Figure 3 This is a flowchart of the intent alignment stage in the four stages of the end-to-end natural language data cockpit generation method described in a specific embodiment; Figure 4 This is a flowchart of the SQL generation stage in the four stages of the end-to-end natural language data cockpit generation method described in the specific embodiment; Figure 5 This is a sequence diagram of multi-source SQL verification in the end-to-end generation method of the natural language data cockpit described in a specific embodiment; Figure 6 This is a flowchart of the template generation stage in the four stages of the end-to-end natural language data cockpit generation method described in the specific embodiment; Figure 7 This is a sequence diagram of the encryption process and anti-tampering verification in the end-to-end generation method of the natural language data cockpit described in a specific embodiment; Figure 8 This is a flowchart of the page arrangement stage in the four stages of the end-to-end natural language data cockpit generation method described in the specific embodiment; Figure 9 This is a schematic diagram of the end-to-end generation system for the natural language data cockpit described in a specific embodiment. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0029] To address the shortcomings of existing technologies in the end-to-end generation process of natural language data dashboards, which fails to meet the comprehensive requirements of ease of use for business users, secure SQL transmission, cross-database dialect adaptation, rapid generation of visualization pages, and operational stability, this application proposes an end-to-end generation method and system for natural language data dashboards based on explicit intent and a multi-stage pipeline. The main inventive concepts include: comprehensively applying technologies such as natural language understanding, database query generation, cross-database dialect adaptation, symmetric encrypted transmission, templated page generation, and offline front-end deployment. This enables the rapid construction of data dashboards, single-chart analysis pages, and thematic data dashboards suitable for enterprise-level business systems, achieving efficient, secure, and accurate conversion from natural language to deployable data visualization pages. The following provides a further detailed description of this application.

[0030] Example 1

[0031] like Figure 1 As shown in the figure, this application discloses an end-to-end generation method for a natural language data dashboard, specifically including: an intent alignment stage, an SQL generation stage, a template generation stage, and a page arrangement stage; wherein each stage is independent and depends on the output of the previous stage, so as to achieve the effect of efficiently, securely, and accurately converting natural language requirements into deployable data visualization pages. Each stage is described in detail below.

[0032] S1. Receive the user's input natural language request, parse the natural language request and convert it into a structured query intent.

[0033] like Figure 2 and Figure 3 As shown, the intent alignment stage includes processes such as parsing, explicitation, disambiguation, and confirmation of natural language requirements. Specific steps include: S1.1 Explicitly transform the received natural language requirements into structured intent objects.

[0034] Specifically, users input natural language requirements, such as optional reference materials like images, PDFs, and Excel files; the received natural language requirements are then explicitly transformed into structured intent objects, such as explicitly structured JSON data (IntentSummary). The IntentSummary object contains several key fields, such as: a data scope field (enumeration: global|per-project|mixed) used to define the data range of the entire page, which clarifies whether the data is global, project-specific, or mixed; a chart binding array (array) used to describe the binding relationship between each chart and the data source: chartBindings[] (array): each element contains fields such as chartId, scope, dependsOn[], paramsFrom[], reason, etc., clearly indicating the data source corresponding to each chart; an ambiguity array (array) used to record ambiguities that need to be resolved: ambiguity[] (array): ambiguities that are not clear to the user and cannot be determined by the knowledge base must all be resolved: true before proceeding to the next stage; and a semantic confirmation array (array) used to record semantic items that need to be confirmed: semantic confirmations[] (array): semantic confirmation items for each calculation / grouping logic must all be confirmed or modified before proceeding to the next stage.

[0035] S1.2. Actively resolve ambiguities in the structured intent object based on the pattern information of the target database and predefined business rules.

[0036] Specifically, proactive disambiguation is achieved through the schema fact-first principle: for identified potential ambiguities, the knowledge base is queried first for inference, and the corresponding item is added to the ambiguity array only when the knowledge base cannot provide a clear answer. For example, before adding an item to the ambiguity list, the knowledge base (tables.json, field_mapping.json, and table_relationships.json) must be explicitly queried first, and the ambiguity is only submitted to the user if none of the three provide a clear answer.

[0037] S1.3. Through at least one round of interactive dialogue with the user, resolve ambiguity in the ambiguity array and confirm or correct items in the semantic confirmation array. For example, after IntentSummary is strictly validated by validateIntentSummary(), it enters a two-round confirmation dialogue with the user: data range confirmation and semantic confirmation.

[0038] S1.4 After all items in the ambiguity array have been resolved and all items in the semantic confirmation array have been confirmed or corrected, a structured query intent containing a clear data range and binding information is generated based on the structured intent object.

[0039] S2. Based on the structured query intent and the schema information of the target database, generate at least one database query statement and its corresponding visual specification description, and form a query mapping object. At the same time, start a multi-source SQL validator to verify the at least one database query statement.

[0040] like Figure 2 and Figure 4 As shown, the SQL generation stage includes operations such as generating query mapping files and validating database query statements. Specific steps include: S2.1 Generate a query mapping file based on the chart binding information in the structured query intent.

[0041] Specifically, a query mapping file is generated based on the chart binding information in the structured query intent, such as generating sqlMap.json based on chartBindings in IntentSummary. Each query mapping in this file includes: the database query statement, the visualization specification description, and data dependencies. The visualization specification description defines the chart type and numerical fields; the data dependencies declare the data source and parameter source information required to execute the query, ensuring that data dependencies across multiple chart pages are explicitly declared and correctly arranged.

[0042] For example, the spec sub-object (visual specification description) of each sqlMap entry defines the metadata required for chart rendering (chartType, valueField, unit, thresholds, and enumMapping, etc.); among which, the enumMapping field is used for enumeration value mapping, which is automatically injected from the knowledge base field_mapping.json or supplemented by the developer if it is not defined in the knowledge base; each sqlMap entry can declare a depends sub-object (data dependency relationship), which declares the dependent data source projectList, the parameter traceability triple params:[{from:'projectList', row:0, field: 'ID'}], and enforces the constraint.

[0043] S2.2 Start the multi-source SQL validator to perform validation on the database query statements in the query mapping file.

[0044] Specifically, such as Figure 5As shown, after the sqlMap is generated, a multi-source SQL validator is started to verify the database query statements in the query mapping file. The verification process includes: dual-source execution verification, incremental retest verification, error handling and prompting, and incremental result comparison.

[0045] Dual-source execution verification verifies the correctness of the same database query statement through at least two different execution channels. These channels include a first channel directly connecting to the target database and a second channel calling a backend application interface. For example, simultaneously calling a direct database connection (MySQL via the mysql2 driver, Oracle via the oracledb driver, and DM via a mysql2-compatible driver) and a backend HTTP API (api / act / nl2sql / execute) to execute the SQL, with the database result as the primary source and the API result as a backup. For databases without a native mysql2 driver, such as Oracle / DM, a 5-second timeout is automatically set to fallback to API verification. Incremental retry verification involves multiple rounds of verification, with each subsequent round only targeting database queries that failed in the previous round. For example, Round 1 performs a full verification, and failed queries are retried in Round 2, with a maximum of 5 rounds. Each round only verifies queries that failed in the previous round, avoiding repeated execution of already successful queries. Error handling and prompts automatically retry database query statements that fail to execute due to network connection errors; and generate corresponding correction prompts based on the type of verification failure; such as automatically matching repair suggestions based on the error type (table does not exist, field does not exist, syntax error, insufficient permissions, etc.) and prompting developers to make corrections.

[0046] Incremental result comparison involves comparing the current verification result with the historical verification result and marking the database query statements whose status has changed. The previous run result is loaded from sqlmap.last_run.json, and each SQL statement is marked with the [FIXED], [CACHED], or [REGRESSION] status.

[0047] In addition, to further ensure the correctness and compatibility of database query statements, such as Figure 4 As shown, the SQL generation stage also includes the following steps: S2.0: Establish an extensible database dialect abstraction layer, which defines a unified syntax conversion interface.

[0048] Specifically, an extensible Dialectect abstraction layer is established to shield the dialect differences between different databases through a unified interface. The dialect abstraction layer defines a unified SqlDialect abstract class, which includes the following methods: quote() (identifier escaping), limit() / applyPagination() (pagination), concat() (string concatenation), now() (current time), ifNull() (null value replacement), dateSub() (date subtraction), substring() (string slicing), castToInt() (type conversion), getMetadataQuery() (metadata query), and stripPagination() (remove pagination clause).

[0049] In step S2.1 above, when generating the query mapping file, an intermediate query description independent of the database dialect is generated based on the structured query intent and the schema information of the target database. Depending on the type of the target database, a corresponding dialect implementation is selected from the database dialect abstraction layer. Using the selected dialect implementation, the intermediate query description is converted into a database query statement conforming to the syntax of the target database and then filled into the query mapping file. The selection of the corresponding dialect implementation from the database dialect abstraction layer includes intelligent conversion logic for handling the database's preset syntax rules. The following examples using several MySQL, Oracle, and DM (DaMeng) dialects illustrate the specific content of the intelligent conversion logic.

[0050] For Oracle database dialect implementations, since Oracle does not support the LIMIT n OFFSET m syntax, its pagination logic is implemented by constructing multi-level nested subqueries and includes an idempotency check mechanism to ensure that multiple pagination operations do not lead to syntax errors. For example: setting SELECT * FROM (SELECT t.*, ROWNUM rn FROM (...original SQL...) t WHERE ROWNUM<= end) WHERE rn>= start; OracleDialect.stripPagination() intelligently identifies this three-level structure through character stream matching and extracts the original SQL, ensuring the idempotency of applyPagination() (multiple calls will not stack pagination clauses).

[0051] For the dialect implementation of the DM database, DM database has several custom differences from the SQL standard: its reserved word processing logic adopts a hierarchical strategy: the first-level reserved word set is used for explicit quotation mark escaping of identifiers; the second-level reserved word set is used for column name-level pattern matching and replacement in the entire SQL string, and avoids escaping SQL syntax keywords. For example, in DM, column name-level reserved words such as DATE, USER, INDEX, KEY, DEL must be enclosed in double quotes ("DATE"), while SQL syntax keywords (SELECT / FROM, etc.) cannot be quoted; a dual-set strategy is adopted: DM_RESERVED (52 words, used for explicit identifier quotation scenarios, such as ORM framework column name concatenation) and DM_COL_RESERVED (13 words, used for applyDialect regular expression replacement to operate on the entire SQL string).

[0052] Step S2.2 above also includes: when the multi-source SQL validator performs validation, it calls the database dialect abstraction layer to perform syntax compliance preprocessing on the database query statement related to the target database. For example: LIMIT syntax conversion: automatically converting MySQL-style LIMIT n or LIMIT m, n to the DM standard; DM does not support TO_CHAR(date, 'YYYY-MM') and DATE_FORMAT, so SUBSTR(CAST(date AS VARCHAR), 1, 7) should be used; CURDATE() is replaced with TRUNC(SYSDATE); date difference is replaced with (date1-date2) by direct subtraction.

[0053] S3. After the at least one database query statement is verified, a front-end page template is generated according to the visualization specification description, the at least one database query statement is encrypted, and the encrypted query statement is injected into the front-end page template to generate a preview page.

[0054] like Figure 2 and Figure 6 As shown, the template generation stage includes steps such as generating front-end page templates, encrypting database query statements, injecting query statements, and multi-level verification. Specific steps include: S3.1. Generate the front-end page template based on a predefined, minimized code skeleton containing core code interaction and data acquisition logic.

[0055] Specifically, the minimized code skeleton includes essential functionalities such as a login pop-up, token acquisition, login, and query execution, ensuring the basic functionality of the page. For example, template.html is generated with the minimum skeleton function "getMinimalSkeleton()" as the mandatory starting point, prohibiting the writing of login / query functions from scratch. The skeleton contains key functions such as the login pop-up (#login-overlay), getToken() (with concurrency protection), doLogin() (compatible with three token response formats), and executeQuery() (with automatic refresh upon token expiration).

[0056] In addition, the generated front-end page template includes a data loading execution function. This data loading execution function is generated based on all data dependencies in the query mapping file, and its execution logic is orchestrated to follow preset dependency constraints. Static analysis is performed on the data loading execution function to identify and block execution logic that violates these dependency constraints.

[0057] Specifically, the preset dependency constraints include: sequence constraints, parameter binding constraints, and execution optimization constraints. The sequence constraint means that for queries with dependencies, the execution of the dependent query completes before the execution of the dependent query begins. The parameter binding constraint means that the parameters required for the execution of the dependent query must be extracted and bound from the result data returned by the query it depends on, based on the data source, row index, and field names specified in the source information list. The execution optimization constraint means grouping queries without direct or indirect dependencies to achieve parallel execution within a group and sequential execution between groups.

[0058] For example, the template `loadAllData()` function must orchestrate queries according to the following rules: Loading order: The dependent party must not issue `executeQuery` before the data of the dependent item is returned; Parameter sourcing: The dependent party's parameters must be precisely extracted from the data returned by the dependent item according to {from, row, field}; Dependency-based grouping and parallelism: All SQL queries are grouped according to dependencies, with parallelism within a group (`Promise.all`) and serial execution between groups, avoiding incorrect parallel / serial combinations. Violating the above constraints will not produce syntax / network errors, but will result in silent failure after data is returned (blank chart or displaying "No data"), making debugging difficult. By statically analyzing the `loadAllData()` function through a rule engine, serial awaits and unimplemented parameter extraction logic can be identified and blocked during the generation phase.

[0059] S3.2. Encrypt the at least one database query statement and inject the encrypted query statement into a specified position of the front-end page template.

[0060] Specifically, an AES-128-CBC+HMAC-SHA256 Encrypt-then-MAC symmetric encryption scheme is established to encrypt the transmission of SQL to prevent business logic leakage and ciphertext tampering.

[0061] like Figure 7 As shown, a random initialization vector is generated, such as: each encryption call crypto.randomBytes(16) generates a 16-byte random IV; using the pre-shared encryption key and initialization vector, the database query statement is symmetrically encrypted to generate ciphertext; such as: AES-128-CBC encryption: the plaintext SQL is encrypted with the pre-shared AES_KEY (16 bytes) and random IV, and padded with PKCS#5 / PKCS#7; based on the pre-shared authentication key, the combination of initialization vector and ciphertext is calculated to generate the message authentication code; such as: HMAC-SHA256 authentication: calculate MAC=HMAC-SHA256(HMAC_KEY, IV||Ciphertext) to obtain a 32-byte authentication tag. The initialization vector, ciphertext, and message authentication code are concatenated and encoded to generate an encrypted query string. For example, the IV, Ciphertext, and MAC segments are concatenated and Base64 encoded to obtain the encryptedSql string. The IV is stored separately in Base64 format for easy segmentation during backend decryption.

[0062] S3.3 Perform multi-level validation on the front-end page template after injecting encrypted query statements.

[0063] Specifically, the multi-level verification includes: placeholder integrity verification, API contract verification, and code security verification. Placeholder integrity verification refers to checking whether the front-end page template contains preset key data and configuration placeholders; for example, the template must contain... , , , , Key placeholders, etc. API contract verification refers to verifying whether the data request logic in the front-end page template conforms to the predefined back-end application interface specifications; for example: checking whether the payload field name of the executeQuery() function is encryptedSql (not ciphertext), whether it contains a description field, whether data.error is handled, whether data.refreshToken == 1 is handled for automatic refresh, and whether the Authorization header lacks the Bearer prefix, etc. Code security verification refers to detecting whether there are predefined categories of coding defects or security risks in the front-end page template through static analysis; for example: detecting 6 common errors through regular expression and context analysis: executeQuery without error checking, await executeQuery without .catch(), rows.map without empty data protection, echarts.init without DOM checking, chart reloading without dispose, and async functions without try-catch.

[0064] S3.4 After successful verification, a preview page is generated based on the front-end page template and the injected encrypted query statement. For example, read template.html, call inject.js to inject encrypted SQL, chart configuration, API address, etc. through placeholders, and generate page_preview.html.

[0065] S4. Perform end-to-end verification on the generated preview page; after successful verification, perform offline deployment.

[0066] Specifically, the page orchestration phase includes end-to-end validation and offline deployment. The specific steps are as follows: S4.1 Perform end-to-end verification on the generated preview page.

[0067] like Figure 6 and Figure 7 As shown, the end-to-end verification steps include: During front-end page runtime, the encrypted query statement string and its corresponding initialization vector are submitted to the back-end application interface. Upon receiving the data, the back-end performs security decryption and query operations to verify the correctness of the message authentication code. If verification fails, the request is rejected. If verification passes, the ciphertext is decrypted using the encryption key and initialization vector to reconstruct the database query statement. The reconstructed database query statement is then executed using a parameterized query method. For example, `batchVerifyHtml()` is called to verify the API execution results of all SQL statements on the page in parallel; if any fails, the verification is blocked.

[0068] S4.2 After successful verification, proceed with offline deployment.

[0069] like Figure 7 As shown, the offline deployment process includes: scanning the preview page to identify and extract all external network resource addresses referenced therein; for example: scanning HTML for... <script src>、<link href>、<style>内url()、@import等多种引用形式，提取所有https: / / 形式外部 URL。对于所提取的外部网络资源地址，执行下载操作并将其保存至本地指定的资源目录中；如：重命名本地化：将jsdelivr风格URL（如 https: / / cdn.jsdelivr.net / npm / echarts@5.4.3 / dist / echarts.min.js）映射为本地文件名（如 echarts@5.4.3.min.js）；其中，对于层叠样式表文件，在下载后进一步扫描其内容以递归下载其所引用的关联资源；如：通过HEAD请求检查CDN资源状态码，200下载、304跳过已存在本地副本，CSS文件下载后再次扫描其内联 url()与@import，递归下载相关资源；将预览页面中对外部网络资源地址的引用，替换为指向本地资源目录中相应文件的相对路径；如：将HTML中的CDNURL替换为相对路径. / ExtJs / 文件名；将更新引用后的页面文件与本地资源目录同步至目标部署位置，生成支持独立运行的数据可视化页面；如：禁止向技能目录写入，防止误操作污染源代码；将page.html与 ExtJs / 资源同步到acm_www / public / 静态站点目录。

[0070] 通过采用上述采用多阶段的处理流程步骤，意图对齐阶段把握用户需求，消除歧义；SQL生成阶段确保数据库查询语句的正确性和兼容性；模板生成阶段保证页面的安全性和规范性；页面编排阶段提高页面的稳定性和可用性。各阶段的严格验证和处理机制，有效解决现有技术中存在的业务人员易用性、SQL安全传输、跨数据库方言适配、可视化页面快速生成、运行稳定性等问题，为用户提供了一种高效、安全、准确的自然语言到数据可视化页面的转换方案，大大提升数据可视化的效率和质量，降低使用门槛和开发成本。

[0071] 实施例2与上述实施例1不同之处在于，为进一步保障上述实施例中各阶段全流程的鲁棒性与自愈能力，通过构建内生的验证反馈闭环，在SQL生成器与下游执行器之间，构建了一个可微分的（或基于模拟的）全局反馈回路，从而保障SQL语法正确、语义一致，还便于下游模板引擎映射、便于编排引擎抽取联动逻辑、便于安全加密传输，且较强的容错性。所述方法还包括：构建内生的验证反馈闭环。该验证反馈闭环包括：离线训练侧微调步骤与在线推理侧迭代修正步骤，通过以上两步骤提升SQL生成结果对模板生成与页面编排的全局友好度。具体包括:针对离线训练侧微调步骤包括：获取历史驾驶舱生成数据；所述历史数据包含自然语言需求、目标数据库Schema、生成的SQL以及对应的模板生成与编排成功 / 失败标记。

[0072] 构建影响预测器；该影响预测器用于在给定自然语言需求、目标数据库Schema和候选SQL时，预测该候选SQL进入候选模板生成阶段和页面编排阶段后的友好度评分及潜在错误。其中，所述友好度评分至少包括可模板化程度（字段是否可直接映射到图表横轴 / 纵轴 / 图例，是否缺失关键数据类型）、可编排程度（是否存在可用于联动的公共维度，过滤条件是否能自动转为参数）、加密适应性（SQL中是否包含无法参数化的明文敏感值）和预估执行风险（通过EXPLAIN计划扫描行数，评估在离线环境执行的可行性）。

[0073] 利用历史驾驶舱生成数据，训练构建好的影响预测器。

[0074] 针对在线推理侧迭代修正步骤包括：利用构建好的影响预测器对同一自然语言需求下生成的多个候选SQL进行评分，得到各维度评分和潜在错误；将影响预测结果作为修正提示，注入SQL生成模型的上下文，触发重写，直至评分达标或达到最大重试次数，进入模板生成。

[0075] 如图8所示，本申请实施例公开一种自然语言数据驾驶舱端到端生成系统，具体包括：意图对齐模块100，用于接收用户输入的自然语言需求，将所述自然语言需求解析并转化为结构化查询意图；所述结构化查询意图至少包括待生成图表的数据范围及绑定信息；SQL生成模块200，用于基于所述结构化查询意图及目标数据库的模式信息，生成至少一个数据库查询语句及对应的可视化规格描述，并形成查询映射对象；所述查询映射对象至少关联数据库查询语句、可视化规格描述及数据依赖关系；启动多源SQL验证器对所述至少一个数据库查询语句进行验证；模板生成模块300，用于在所述至少一个数据库查询语句验证通过后，根据可视化规格描述生成前端页面模板，对所述至少一个数据库查询语句进行加密处理，并将加密后的查询语句注入前端页面模板以生成预览页面；页面编排模块400，用于对生成的预览页面进行端到端验证；验证通过后，执行离线化部署处理；所述离线化部署处理包括：将生成的预览页面中所引用的外部网络资源下载至本地，并更新页面中对外部网络资源的引用链接，生成支持独立运行的数据可视化页面。

[0076] 本申请实施例还公开一种计算机可读存储介质。

[0077] 具体来说，该计算机可读存储介质，其存储有能够被处理器加载并执行如上述自然语言数据驾驶舱端到端生成方法的计算机程序，该计算机可读存储介质例如包括：U盘、移动硬盘、只读存储器(ROM)、随机存取存储器(RAM)、磁碟或者光盘等各种可以存储程序代码的介质。

[0078] 本申请实施例还公开一种计算机设备。

[0079] 具体来说，计算机设备包括存储器和处理器，存储器上存储有能够被处理器加载并执行上述自然语言数据驾驶舱端到端生成方法的计算机程序。

[0080] 以上均为本申请的较佳实施例，并非依此限制本申请的保护范围，本说明书（包括摘要和附图）中公开的任一特征，除非特别叙述，均可被其它等效或者具有类似目的的替代特征加以替换。即，除非特别叙述，每个特征只是一系列等效或类似特征中的一个例子而已。< / script>

Claims

1. An end-to-end method for generating natural language data cockpits, characterized in that, include: S1. Receive the user's input in natural language, parse the natural language requirement and convert it into a structured query intent; The structured query intent includes at least the data range and binding information of the chart to be generated; S2. Based on the structured query intent and the schema information of the target database, generate at least one database query statement and its corresponding visual specification description, and form a query mapping object; The query mapping object is associated with at least the database query statement, the visual specification description, and the data dependencies; Start the multi-source SQL validator to validate the at least one database query statement; S3. After the at least one database query statement is verified, a front-end page template is generated according to the visualization specification description. The at least one database query statement is encrypted and the encrypted query statement is injected into the front-end page template to generate a preview page. S4. Perform end-to-end verification on the generated preview page; After successful verification, proceed with offline deployment. The offline deployment process includes: downloading the external network resources referenced in the generated preview page to the local machine, updating the reference links to the external network resources in the page, and generating a data visualization page that supports independent operation; Step S1 includes: S1.1 Explicitly convert the received natural language requirements into a structured intent object; the structured intent object includes at least: a data range field for defining the entire page data range, a chart binding array for describing the binding relationship between each chart and the data source, an ambiguity array for recording the ambiguities that need to be resolved, and a semantic confirmation array for recording the semantic items that need to be confirmed. S1.

2. Based on the pattern information of the target database and the predefined business rules, actively resolve ambiguities of the structured intent object; for the identified potential ambiguities, first query the knowledge base for inference, and only add the corresponding item to the ambiguity array when the knowledge base cannot provide a clear answer; S1.

3. Through at least one round of interactive dialogue with the user, resolve the ambiguity of items in the ambiguity array and confirm or correct items in the semantic confirmation array. S1.4 After all items in the ambiguity array have been resolved and all items in the semantic confirmation array have been confirmed or corrected, a structured query intent containing a clear data range and binding information is generated based on the structured intent object. Step S2 includes: S2.

1. Based on the chart binding information in the structured query intent, generate a query mapping file, and each query mapping in the query mapping file includes at least: a database query statement, a visualization specification description, and data dependencies; wherein, the visualization specification description is used to define the chart type and numerical fields; the data dependencies are used to declare the data source and parameter source information required to execute the query; S2.2, Start the multi-source SQL validator to verify the database query statements in the query mapping file; the verification process includes: dual-source execution verification, incremental retest verification, error handling and prompting, and incremental result comparison; the dual-source execution verification verifies the correctness of the same database query statement through at least two different execution channels; the execution channels include: a first channel directly connected to the target database, and a second channel through calling the backend application interface; the incremental retest verification is performed through multiple rounds of verification, with each subsequent verification round only targeting database query statements that failed in the previous verification round; the error handling and prompting automatically retryes database query statements that fail due to network connection abnormalities; and generates corresponding correction prompts based on the type of verification failure; the incremental result comparison compares the current verification result with the historical verification result and marks the database query statements whose status has changed; step S2 also includes: S2.0, Establish an extensible database dialect abstraction layer; the database dialect abstraction layer defines a unified syntax conversion interface; In step S2.1, when generating the query mapping file, an intermediate query description independent of the database dialect is generated based on the structured query intent and the pattern information of the target database. Step S2.1 further includes: selecting the corresponding dialect implementation from the database dialect abstraction layer according to the type of the target database; using the selected dialect implementation, converting the intermediate query description into a database query statement that conforms to the syntax of the target database, and filling it into the query mapping file; Step S2.2 also includes: when the multi-source SQL validator performs validation, it calls the database dialect abstraction layer to perform syntax compliance preprocessing on the database query statement related to the target database; Step S3 includes: S3.

1. Generate the front-end page template based on a predefined, minimized code skeleton containing core code interaction and data acquisition logic; S3.

2. Encrypt the at least one database query statement and inject the encrypted query statement into a specified position in the front-end page template; S3.3 Perform multi-level verification on the front-end page template after injecting encrypted query statements; the multi-level verification includes: placeholder integrity verification, API contract verification, and code security verification; the placeholder integrity verification refers to checking whether the front-end page template contains preset key data and configuration placeholders; the API contract verification refers to verifying whether the data request logic in the front-end page template conforms to the predefined back-end application interface specification; the code security verification refers to detecting whether there are preset categories of coding defects or security risks in the front-end page template through static analysis; S3.4 After the verification is passed, a preview page is generated based on the front-end page template and the injected encrypted query statement; In step S3.1, the front-end page template includes a data loading execution function. This function is generated based on all data dependencies in the query mapping file, and its execution logic is orchestrated to follow preset dependency constraints. These preset dependency constraints include: order constraints, parameter binding constraints, and execution optimization constraints. The order constraint means that for queries with dependencies, the execution of the dependent query completes before the execution of the dependent query begins. The parameter binding constraint means that the parameters required for the execution of the dependent query must be extracted and bound from the result data returned by the query it depends on, based on the data source, row index, and field name specified in the traceability information list. The execution optimization constraint means grouping queries without direct or indirect dependencies to achieve parallel execution within a group and sequential execution between groups. The static analysis in step S3.3 includes: scanning the data loading execution function to identify and block execution logic that violates the preset dependency constraints; The encryption process in step S3.2 includes: Generate a random initialization vector; use the pre-shared encryption key and initialization vector to perform symmetric encryption on the database query statement to generate ciphertext; calculate the combination of initialization vector and ciphertext based on the pre-shared authentication key to generate a message authentication code; concatenate and encode the initialization vector, ciphertext, and message authentication code to generate the encrypted query statement string; Step S4 includes: S4.1 Perform end-to-end verification on the generated preview page; the end-to-end verification steps include: when the front-end page is running, submit the encrypted query statement string and its corresponding initialization vector to the back-end application interface, so that after the back-end receives the data, it performs security decryption and query operations, verifies the correctness of the message authentication code, and rejects the request if the verification fails; after the verification is successful, use the encryption key and initialization vector to decrypt the ciphertext and restore the database query statement; execute the restored database query statement using parameterized query method; S4.2 After successful verification, perform offline deployment processing; the offline deployment processing includes: scanning the preview page, identifying and extracting all external network resource addresses referenced therein; for the extracted external network resource addresses, performing a download operation and saving them to a specified local resource directory; wherein, for cascading style sheet files, after downloading, further scanning their content to recursively download the associated resources they reference; replacing the references to external network resource addresses in the preview page with relative paths pointing to the corresponding files in the local resource directory; synchronizing the updated page files and the local resource directory to the target deployment location, generating a data visualization page that supports independent operation.

2. The end-to-end natural language data cockpit generation method according to claim 1, characterized in that, Step S2.1 involves selecting the corresponding dialect from the database dialect abstraction layer, which includes intelligent conversion logic for processing the database's preset syntax rules. The intelligent conversion logic includes: for the Oracle database dialect implementation, its pagination processing logic is implemented by constructing multi-level nested subqueries, and includes an idempotency check mechanism to ensure that multiple pagination operations will not cause syntax errors; for the DM database dialect implementation, its reserved word processing logic adopts a hierarchical strategy: the first-level reserved word set is used to explicitly escape identifiers with quotation marks, and the second-level reserved word set is used to perform column name-level pattern matching and replacement in the entire SQL string, and avoids escaping SQL syntax keywords.

3. An end-to-end natural language data cockpit generation system applying the method of any one of claims 1-2, characterized in that, include: The intent alignment module is used to receive the user's input natural language request, parse the natural language request and convert it into a structured query intent; The structured query intent includes at least the data range and binding information of the chart to be generated; The SQL generation module is used to generate at least one database query statement and a corresponding visual specification description based on the structured query intent and the schema information of the target database, and to form a query mapping object. The query mapping object is associated with at least the database query statement, the visual specification description, and the data dependencies; Start the multi-source SQL validator to validate the at least one database query statement; The template generation module is used to generate a front-end page template according to the visual specification description after the at least one database query statement is verified, to encrypt the at least one database query statement, and to inject the encrypted query statement into the front-end page template to generate a preview page. The page arrangement module is used to perform end-to-end validation of the generated preview page; After successful verification, proceed with offline deployment. The offline deployment process includes: downloading the external network resources referenced in the generated preview page to the local machine, updating the reference links to the external network resources in the page, and generating a data visualization page that supports independent operation.

Images