A chart information generation method based on retrieval enhancement generation and natural language processing

Abstract

The application provides a chart information generation method based on retrieval enhancement and natural language processing, and belongs to the technical field of data visualization and natural language processing; features of database fields are extracted, types are identified, and modeling visual roles are built; visual channels of chart templates are parsed and constraints are established; and a multi-source chart knowledge base is constructed. A user query is parsed into a chart intent, data query and chart template candidates are retrieved from the knowledge base, joint retrieval results are obtained through execution verification and counterfactual screening. The matching cost of fields and visual channels is calculated through a neural network, combined with a symbol penalty term, and an optimal transport is used to solve the alignment result. Based on the retrieval result and the alignment result, a context is constructed, a template abstract syntax tree is used as a skeleton to constrain decoding to generate chart code, and after execution, a comprehensive reward is calculated according to the result; if the result fails, the result is repaired and retried until termination, and finally, the chart is output. The method improves the accuracy, executability and reliability of chart generation.

Description

Technical Field

[0001] This invention relates to a method for generating chart information based on retrieval enhancement and natural language processing, belonging to the field of data visualization and natural language processing technology. Background Technology

[0002] With the continuous development of enterprise digital transformation, business intelligence analytics, and data visualization applications, structured data has become widely used in scenarios such as sales management, operational monitoring, financial analysis, medical statistics, industrial production, and government administration. Users want to directly describe their analytical needs using natural language. For example, they might want to statistically analyze monthly sales trends, compare order volumes in different regions, and display the profit margins of various product categories. The system can then automatically select relevant fields from the database, generate query results, and further generate interactive and renderable chart code. Therefore, natural language-based chart generation technology has become an important technological direction for lowering the barrier to data analysis, improving business decision-making efficiency, and enhancing the data understanding capabilities of non-professional users.

[0003] However, in practical applications, generating chart code from natural language is not a simple text generation problem. It involves multiple stages, including natural language understanding, database field matching, SQL query generation, chart template selection, field and visual channel binding, front-end code generation, and runtime environment verification. User queries often have characteristics such as vague expression, omitted indicators, inconsistent field names, and implicit business semantics; databases also have problems such as complex table structures, diverse field types, non-standard field naming, missing data, and multi-table joins; chart templates also have strict type constraints, channel constraints, and syntax constraints. For example, line charts usually require the horizontal axis to have time or ordered category meaning, and the vertical axis to have numerical fields; pie charts require category fields and numerical fields to appear in pairs; scatter plots require two numerical fields to form a two-dimensional space. If the system generates charts solely based on semantic similarity or the generation capabilities of large language models, problems such as incorrect field selection, chart type mismatch, unreasonable visual channel binding, unexecutable code, or empty rendering results are likely to occur.

[0004] Currently, common technical solutions for achieving natural language-driven chart generation mainly fall into the following three categories: (1) Chart generation method based on rule templates and manual configuration: This method usually pre-sets fixed chart templates and field matching rules. When the user enters a query, the system selects the chart type based on keywords, field names, or simple rules, and fills the database fields into the template. This type of method is simple to implement and highly controllable, and is suitable for business report scenarios with standardized field naming and fixed query types. However, this method relies heavily on manual rule design and is difficult to adapt to complex natural language expressions and variable database structures. When the user query contains synonyms, omitted expressions, or cross-table analysis requirements, the rule system often cannot accurately understand the true intent; at the same time, the rule method usually lacks a comprehensive judgment on field statistical characteristics, template structure constraints, and front-end execution results, and is prone to generating charts that are formally matched but actually unusable.

[0005] (2) Chart generation methods based on semantic retrieval and template matching: These methods typically use text encoding models to calculate the semantic similarity between user queries and database fields and chart templates, selecting fields and templates with high similarity for chart generation. Compared to pure rule-based methods, this approach improves semantic understanding and generalization capabilities, and can handle a certain degree of natural language variation. However, these methods are essentially still similarity ranking methods, only able to determine whether queries, fields, and templates are semantically similar, but unable to determine whether the generated SQL can return valid data, or whether the selected chart template is truly suitable for the current data structure. For example, if a template is semantically similar to the user query, but its required field types do not match the fields returned by the current SQL, the chart may still fail to render. Therefore, relying solely on semantic retrieval is insufficient to solve the problem of semantically related but structurally unexecutable data.

[0006] (3) End-to-end chart code generation method based on large language models: With the development of large language models, some methods attempt to directly input user queries, database field descriptions, and chart requirements into the model, and generate SQL, chart configuration code, or front-end visualization code from the end. This method has strong language understanding and code generation capabilities, and shows certain advantages in open chart generation tasks. However, this type of method generally suffers from insufficient controllability and reliability. On the one hand, the large language model may generate field names that do not exist in the database, incorrect aggregation methods, or configuration items that do not conform to template constraints, resulting in field illusions and channel mismatches; on the other hand, although the code generated by the model seems reasonable at the text level, it may have execution problems such as syntax errors, type errors, empty data, DOM nodes not generated, or rendering failures. In addition, end-to-end methods usually lack explicit modeling of the field-visual channel binding process, and also lack an automatic repair mechanism based on sandbox execution results, making it difficult to ensure that the final chart code runs stably in a real front-end environment.

[0007] In summary, existing natural language graph generation methods generally suffer from problems such as strong rule dependencies, lack of execution judgment in semantic retrieval, lack of global constraints in field and visual channel binding, susceptibility to illusions in end-to-end code generation, and lack of automatic error correction mechanisms. Therefore, there is an urgent need to develop a graph information generation method based on retrieval-enhanced generation and natural language processing, capable of establishing a complete, searchable, alignable, executable, and error-correctable link between user natural language queries, relational databases, graph template libraries, and the front-end execution environment. Summary of the Invention

[0008] The purpose of this invention is to provide a chart information generation method based on retrieval enhancement and natural language processing, which improves the accuracy, executability, interpretability and reliability of chart generation in practical applications.

[0009] To achieve the above objectives, the present invention employs the following technical solution: A method for generating chart information based on retrieval enhancement and natural language processing includes the following steps: Acquire data field information and chart template library from the database; perform feature extraction, field type recognition, and visualization role prior modeling on the database fields to obtain field representations; parse the chart templates to extract the visual channels and establish a set of template constraints; construct a multi-source chart knowledge base based on field representations, visual channel representations, and template constraint sets through cross-modal comparison pre-training. The system parses user natural language queries into chart intents; generates data query candidates and chart template candidates based on a multi-source chart knowledge base, and combines them into a joint candidate; performs execution verification on the joint candidate to test its chart execution feasibility; performs counterfactual substitution on the fields involved in the joint candidate, and filters the joint candidate based on the changes in joint utility before and after the substitution to obtain a joint retrieval result set. Based on the joint retrieval result set, the matching cost between the field and the visual channel is calculated through a neural network, and a comprehensive matching cost matrix is ​​obtained by combining a constraint-based symbolic penalty term. The optimal transmission method is used to solve the alignment result between the field and the visual channel, forming a field-visual channel alignment result set. Based on the joint retrieval result set and the field-visual channel aligned result set, a code generation context representation is constructed. The abstract syntax tree of the chart template is used as the generation skeleton for constraint decoding to generate chart code. The chart code is executed, and the comprehensive reward is calculated based on the execution result. If the execution fails, the chart code is repaired based on the error message, and the check and execution are repeated until the termination condition is met, and the chart generation result is output.

[0010] Preferably, the feature extraction includes semantic feature extraction, statistical feature extraction, and structural feature extraction, specifically including: Read the table name, field name, field comments, and field information of the same table, and perform joint encoding through a text semantic encoder based on a pre-trained language model to obtain semantic features; Statistical analysis is performed on the actual data records of the fields to obtain statistical characteristics. Among them, the statistical characteristics of numerical fields include the range of values, mean, standard deviation, quantiles and missing rate; the statistical characteristics of categorical fields include the number of unique values, category entropy and the proportion of high-frequency categories; and the statistical characteristics of time fields include the time span and time granularity. Based on the table, field, primary key, and foreign key information in the database schema, a database schema graph is constructed using fields as graph nodes and primary key associations and foreign key references between fields as graph edges. Using the database schema graph as input, a graph neural network is used to aggregate neighborhood information of field nodes to obtain structural features. After mapping semantic features, statistical features, and structural features to a unified representation space through their respective trainable mapping matrices, they are concatenated and normalized to obtain the field representation. The field type identification and visualization of role prior modeling specifically includes: Based on the database declaration type and combined with the value form and data format of the actual data records in the field, the fields are divided into numeric, categorical, time, boolean, and text types, forming a set of field types; Visual role priors are established based on field types and their statistical characteristics. Time-type fields are given priority as candidates for the horizontal axis; numerical fields are given priority as candidates for the vertical axis; low-cardinality categorical fields are given priority as candidates for color, series, or legend; and high-cardinality categorical or text fields are given priority as candidates for labels or tooltips. The visual role priors of all fields constitute the visual role prior set. The chart template is parsed to extract the visual channels and a set of template constraints is established, specifically including: Read the chart configuration code of the templates in the chart template library, unify the data field configuration format and configuration item name of the templates, and delete redundant style configurations to obtain standardized templates, thus forming a standardized template library; The standardized template is parsed into an abstract syntax tree according to the syntax rules of the template configuration language. The nodes that can be bound to database fields are located according to the predefined configuration item names. The visual channels of the horizontal axis, vertical axis, series, color, legend, label and tooltip are extracted from the abstract syntax tree to obtain the visual channel representation set of the template. Visual channel constraints are established based on the actual data mapping relationships in the template, forming a set of visual channel constraints for the template. The visual channel constraints include the constraints between each visual channel and between the visual channel and the field type, number of fields, and data organization method.

[0011] Preferably, the construction of the multi-source chart knowledge base specifically includes: By utilizing historically generated chart records, the fields actually used and their corresponding visual channels are used as positive samples, and negative samples are constructed by replacing fields of the same type but different semantics and replacing fields that violate template channel constraints with visual channels. The semantic similarity between the field representation and the visual channel representation is calculated, and the positive and negative samples are used for comparative learning to make the successfully bound field-channel combination obtain high similarity, and the field-channel combination that does not match semantically or violates the channel constraints obtain low similarity, thereby training to obtain the field mapping matrix and the visual channel mapping matrix. The field representations, visual channel representations, template constraint sets, and the trained field mapping matrices and visual channel mapping matrices are stored uniformly to form the multi-source chart knowledge base.

[0012] Preferably, the step of parsing the user's natural language query into a chart intent specifically includes: The user's natural language query is parsed into a query intent graph. The nodes in the query intent graph include indicators, grouping dimensions, filtering conditions, aggregation methods, time granularity, and chart structure preferences. The edges represent the modification or dependency relationships between the intent elements. The query intent graph is encoded by a language model and propagated through a graph neural network to obtain a query intent representation. The graph neural network consists of a node input mapping layer, two graph attention network layers, and a global readout layer. The node input mapping layer maps the language encoding features of each node to a unified dimension. The first graph attention network aggregates the first-order neighbor information of each node. The second graph attention network aggregates the second-order neighbor information based on the output of the first layer. The global readout layer aggregates the representations of all nodes to obtain a global representation of the query intent.

[0013] Preferably, the step of generating data query candidates and chart template candidates based on a multi-source chart knowledge base specifically includes: Calculate the semantic similarity between the query intent representation and each field representation in the field representation set, sort them from high to low similarity and retain the top n fields as candidate fields, generate b data query candidates based on the candidate fields, and each data query candidate includes the SQL statement, the database fields used and the corresponding aggregation, filtering and grouping information; Calculate the matching similarity between the query intent representation and the visual channel representation corresponding to each chart template, and perform constraint filtering on the candidate templates in combination with the template constraint set. After sorting according to the comprehensive matching score, retain the top k templates as template candidates. The data query candidates and template candidates are combined to form a SQL-template joint candidate set. In cases where the user explicitly specifies a chart type but it conflicts with the form of the query data, the user-specified template is retained as a candidate and its potential constraint conflicts are judged, while other compatible templates are allowed to enter the candidate set.

[0014] Preferably, the step of performing verification on the joint candidates specifically includes: For each SQL-template union candidate, execute the corresponding SQL in a read-only, time-limited, and quantity-limited controlled database environment to obtain a query result sub-table. The union candidate is marked as unexecutable when any of the following conditions occur: SQL execution fails; query result is empty; SQL reference field does not exist; field type is incompatible with template requirements; query result size exceeds a preset security threshold; or it involves unauthorized data fields. For joint candidates executed via SQL, based on the visual channel configuration defined in the template, the corresponding fields in the query result sub-table are filled into the horizontal axis, vertical axis, series, and color configuration items to generate a lightweight chart sketch without complex styles and custom scripts. The sketch is then checked sequentially based on the template constraint set: whether the required visual channels are satisfied, whether the field types meet the constraints, whether the data is not null, whether the cardinality of the category field is within the range that the template can handle, and whether the horizontal axis, vertical axis, and series configurations can form a valid chart structure. The sketch is considered to have passed the verification only if all of the above checks are satisfied. For joint candidates that pass both execution and sketch validation, the execution utility is calculated based on the SQL execution status, the data integrity of the query results, and the structural compliance of the sketch. The joint utility is then updated based on the execution utility. Joint candidates that fail validation are eliminated. The formula for updating the joint utility is as follows: , , , in, For the purpose of execution; This indicates the SQL execution status. It is set to 1 when the SQL executes successfully in a controlled database environment without any permission, timeout, or field errors; otherwise, it is set to 0. This indicates the data integrity of the query results. It is set to 1 if the data is complete and meets the field type requirements, and 0 otherwise. This indicates whether the sketch meets the diagram structure requirements. It is set to 1 if it meets the visual channel and structural constraints specified in the template, and 0 otherwise. , and For the corresponding weights; This indicates the combined utility after execution and verification. For initial joint utility; Utility weights for execution; Indicates SQL-template combined candidate The initial joint utility; The semantic matching utility between query intent and SQL candidates and chart templates is obtained by calculating cosine similarity. This indicates the structural adaptation effect between the fields involved in the SQL and the template visual channel; Indicates the corresponding effect weight; This indicates the query intent; SQL candidate; For template visual channel representation; This represents the c-th icon template for the recall. This is the b-th data query candidate; A collection of field types; For template constraint set; This represents the sub-table of the query results obtained by executing the b-th SQL query candidate; The calculation method is as follows: Based on the set of field types, the set of visual channel representations, and the set of template constraints, check one by one whether the field type, field quantity, and field organization method in the SQL candidate meet the visual channel requirements corresponding to the template. If they meet the requirements, record 1; otherwise, record 0. The total field type matching result, the total field quantity matching result, and the total data organization method matching result are weighted and summed to obtain the structure adaptation utility.

[0015] Preferably, the step of performing counterfactual substitution on the fields involved in the joint candidates, and filtering the joint candidates based on the change in joint utility before and after the substitution, to obtain a joint retrieval result set; specifically including: For the validated SQL-template joint candidate set, select fields with the same type as the field to be tested but different semantics as replacement candidates. Perform counterfactual replacements on the indicator, grouping dimension and time fields in the query, regenerate and execute the corresponding SQL to obtain the counterfactual joint utility. Calculate the field replacement sensitivity, which is the change in combined utility before and after the field replacement; The validated joint candidates are weighted and sorted based on the updated joint utility and field replacement sensitivity. The joint candidate with the highest score, its corresponding final SQL query, final chart template, final data sub-table, and the joint utility changes resulting from the counterfactual replacement of the key fields corresponding to the joint candidate with the highest score are obtained to form a joint retrieval result set.

[0016] Preferably, the comprehensive matching cost matrix is ​​obtained in the following way: A multilayer perceptron matching network is used to fuse the features of the field representation and the visual channel representation, outputting the matching score between the field and the visual channel, and the negative value of the matching score is converted into a neural matching cost. A symbolic penalty term is constructed, which includes: a field type compatibility penalty, which is 0 when the field type meets the field type requirements of the visual channel, and 1 otherwise; a category cardinality compatibility penalty, which is 0 when the field cardinality is within the range allowed by the visual channel, and 1 otherwise; and a template constraint penalty, which is 0 when the field and the visual channel meet the corresponding field source, syntax, and configuration constraints in the template constraint set, and 1 otherwise; each penalty is multiplied by its corresponding non-negative penalty weight. The neural matching cost is added to the symbolic penalty term to obtain the field-visual channel matching cost. The field-visual channel matching costs of the database fields are combined to obtain the comprehensive matching cost matrix. For critical fields whose sensitivity to field replacement exceeds a preset threshold, the cost of field-visual channel matching is... as follows: , in, For the cost of neural matching, For symbolic penalty terms, Adjust the weights for sensitivity. The joint utility change resulting from counterfactual replacement of the key field corresponding to the highest-scoring joint candidate.

[0017] Preferably, the alignment results of the field and the visual channel are obtained by using the optimal transmission solution to form a field-visual channel alignment result set; the specific method is as follows: Based on the constraint information in the template constraint set corresponding to the final chart template, determine the field requirement vector for each visual channel. and the capacity vector of each field The capacity vector represents the maximum number of visual channels that a field can be bound to. In the field requirement vector, the required visual channel is 1, and the optional visual channel can be satisfied by an empty field node. The optimization objective is constructed by weighting the total cost of field-visual channel matching, the entropy regularization term, and the structure penalty term. The optimization objective is as follows: , , in, This represents the cost of matching each field-visual channel. Corresponding soft binding probability The total cost of field-visual channel matching is obtained by performing element-wise multiplication and summing. Represents the entropy regularization term; Indicates the structural penalty term; Represents the entropy regularity coefficient; Indicates the structural penalty weight; For field-visual channel soft binding matrix; This is the set of template constraints corresponding to the final chart template; This represents the visual channel combination relationship within the template constraint set; Represents the constraint weights of the corresponding channel relationships; A compatibility function representing the binding states between two visual channels; Indicates the relationship with the first The binding status of another visual channel that has a template combination constraint relationship with one visual channel. For the prior set of visual characters; The optimal soft binding matrix for the field-visual channel was obtained by solving the optimization objective using the Sinkhorn iteration method. ; Using the binding probabilities of each field in the optimal soft binding matrix as candidate matching weights, the field-visual channel combination with the highest total binding probability is selected, and combinations that violate the template constraints are eliminated during the selection process. Then, the field capacity, required visual channels, field type, category cardinality, and visual channel combination relationship are checked in turn. If there is a constraint violation, the corresponding binding is canceled and reselected from the remaining candidates until a discrete field-visual channel binding matrix that satisfies all constraints is obtained. For each visual channel, the binding confidence is calculated based on the optimal soft binding matrix. When the highest binding probability is lower than the preset confidence threshold, or the difference between the highest binding probability and the second highest binding probability is lower than the preset interval threshold, it is determined that there is binding ambiguity in the visual channel. A preset number of candidate fields that meet the template constraints are retained, and a set of low-confidence candidate fields is established by combining the replacement sensitivity of the corresponding fields. The optimal soft binding matrix, the discrete field-visual channel binding matrix, the binding confidence, and the low-confidence candidate field set are used as the field-visual channel alignment result set.

[0018] Preferably, the chart generation result is generated in the following specific way: The code generation context is constructed, including: user query, actual field names and field types in the final data sub-table, final chart template, discrete field-visual channel binding matrix, low-confidence candidate field set, template constraints, and target chart library version information; wherein, only fields that actually exist in the final data sub-table are provided as generation candidates, and adjustments to candidate fields are only allowed from the low-confidence candidate field set; The constraint decoding uses the abstract syntax tree of the final chart template as its skeleton, and generates data source, horizontal axis, vertical axis, series, color, legend and tooltip configuration node by node according to the path of the abstract syntax tree; in each decoding step, a joint mask is constructed based on the abstract syntax tree of the final chart template, field binding results and field type information, and the constraint decoding probability is determined based on the joint mask. After all the abstract syntax tree nodes are decoded, the chart code is reconstructed. The chart code is injected into an isolated sandbox for execution. The isolated sandbox restricts network access, CPU, memory and execution time, and only loads the target chart library version, resulting in a sandbox execution result that includes execution status, error logs, rendering results and chart quality information. If the code executes successfully, a comprehensive reward is calculated based on the sandbox execution results. The comprehensive reward is obtained by weighted summation of the correctness of the code syntax, whether the chart is successfully rendered, the chart layout and readability, and the consistency between the chart and the user's query intent. If the code execution fails or the chart fails to render correctly, the error log returned by the sandbox will be converted into structured error information. The structured error information includes the error category, the corresponding abstract syntax tree node path, the fields involved, and the alternative candidate fields. Based on structured error information, code repair is modeled as a Markov decision process, where the state includes the current abstract syntax tree node, error information, field bindings and historical repair trajectory, and the actions include replacing candidate fields, supplementing required nodes and deleting invalid nodes. The policy network adopts a near-end policy to optimize and update the repair policy. After each patch is completed, static checks and sandbox execution are performed again. The patching stops when the chart code executes successfully and the overall reward reaches the quality threshold. The final executable chart code is output, and a chart generation result is generated that records the final executable chart code, sandbox execution results, discrete field-visual channel binding matrix, and the abstract syntax tree nodes, patch actions, reward changes, and termination reasons for each round of patching.

[0019] The advantages of this invention are as follows: For natural language-driven chart generation tasks, this invention no longer treats data retrieval and template retrieval as two independent similarity ranking tasks. Instead, it combines SQL candidates and chart template candidates into joint candidate pairs and uses lightweight sketch execution to determine whether the candidate combination can generate a valid chart. At the same time, it introduces a counterfactual field intervention mechanism, requiring that the chart intent satisfaction will be significantly reduced after the key field is replaced by a similar but different semantic field, thereby achieving joint screening of semantic relevance, structural executability, and the necessity of key fields.

[0020] This invention models field-visual channel binding as a globally optimal transmission problem. Based on the neural network calculation of the matching cost between fields and visual channels, it integrates field type constraints, field capacity constraints, channel requirement constraints, template syntax constraints, and higher-order chart structure factors. Through differentiable Sinkhorn iteration, a soft alignment matrix is ​​obtained and further projected into discrete binding results that satisfy template constraints, thereby avoiding the channel mismatch and unreasonable chart structure problems caused by traditional local scoring methods.

[0021] This invention introduces abstract syntax tree state constraints, field validity constraints, type constraints, and field-channel alignment probability bias during the chart code generation process to impose fine-grained restrictions on the generated tokens. After generation, it further obtains error types, error logs, DOM summaries, and rendering quality indicators through static verification and isolated sandbox execution, and generates patch-based repair schemes based on the error AST path, thereby achieving automatic verification and automatic repair of the chart code and improving the executability, renderability, and actual deployment reliability of the generated results. Attached Figure Description

[0022] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0023] Figure 1 This is a flowchart of the overall technical roadmap.

[0024] Figure 2 This is a network structure diagram of a graph information generation model based on retrieval enhancement and natural language processing.

[0025] Figure 3 For field-visual channel matching accuracy analysis.

[0026] Figure 4 A sales trend chart automatically generated based on natural language commands. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] This invention presents a chart generation method based on retrieval enhancement and natural language processing. Addressing the task of automatically generating charts from structured data driven by natural language, it establishes a learnable, constrained, executable, and error-correctable chart generation chain between user queries, relational databases, chart template libraries, and the front-end rendering environment. Unlike methods that handle SQL generation, template selection, field binding, and code repair separately, this invention unifies query intent understanding, data retrieval, template adaptation, field-visual channel alignment, and code correction into a joint optimization process involving neural models and symbolic rules. This reduces problems such as query result mismatch with templates, field reuse conflicts, chart syntax errors, unexecutable code, and uncontrollable repair processes, ultimately generating executable, renderable charts that align with user query intent. Figure 1 The chart code.

[0029] like Figure 1 - Figure 2 As shown, a method for generating chart information based on retrieval enhancement and natural language processing includes the following steps: S1: Obtain data field information and chart template library from the database; perform feature extraction, field type recognition, and visualization role prior modeling on the database fields to obtain field representations; parse the chart templates to extract the visual channels and establish a set of template constraints; construct a multi-source chart knowledge base based on field representations, visual channel representations, and template constraint sets through cross-modal comparison pre-training; S2: Parse the user's natural language query into a chart intent; generate data query candidates and chart template candidates based on a multi-source chart knowledge base, and form a joint candidate; perform execution verification on the joint candidate to test the chart execution feasibility of the joint candidate; perform counterfactual substitution on the fields involved in the joint candidate, and filter the joint candidate according to the change in joint utility before and after the substitution to obtain a joint retrieval result set; S3: Based on the joint retrieval result set, the matching cost between the field and the visual channel is calculated through a neural network, and a comprehensive matching cost matrix is ​​obtained by combining a constraint-based symbolic penalty term. The optimal transmission is used to solve the alignment result between the field and the visual channel, forming a field-visual channel alignment result set. S4: Construct a code generation context representation based on the joint retrieval result set and the field-visual channel aligned result set. Use the abstract syntax tree of the chart template as the generation skeleton to perform constraint decoding and generate chart code. Execute the chart code and calculate the comprehensive reward based on the execution result. If the execution fails, repair the chart code based on the error message and repeat the check and execution until the termination condition is met, and output the chart generation result.

[0030] As a refinement of the above embodiments, step S1 is a process of constructing and unifying the representation of a multi-source graph knowledge base, which includes: Construction of multimodal information in field S1-1 First, the target relational database is read based on the user-specified data source to obtain the database field set. , This represents the i-th database field.

[0031] Read The semantic features of a field are constructed by including its table name, field name, field comment, database declaration type, and information about other fields in the same table. Statistical characteristics and structural features The semantic features consist of the table name to which the field belongs, the field name, the field comment, and information about other fields in the same table, and are encoded into a semantic feature vector. .

[0032] in, This represents a text semantic encoder based on the pre-trained language model BERT, used to jointly encode field names, table names, field comments, and context within the same table to obtain the semantic feature vector of the field.

[0033] Statistical characteristics are directly applied to this field by the system. Statistical analysis is performed on actual data records. This includes numerical and categorical fields; numerical fields include value range, mean, standard deviation, quantiles, and missing rate; categorical fields include the number of unique values, category entropy, and percentage of high-frequency categories; and time fields include time span and time granularity.

[0034] The structural features are obtained by the system through statistical calculations of the actual data records of the fields. Furthermore, the primary key and foreign key information in the database schema is read, and a database schema diagram is constructed based on the foreign key reference relationships between tables. Specifically, primary keys are used to uniquely identify records in a data table, while foreign keys are used to reference the primary keys of other data tables to establish relationships between them. Therefore, the edges in the database schema graph are mainly composed of foreign key relationships, representing the connections between different data tables and their fields. The system takes the database schema graph as input and uses a graph neural network to aggregate neighborhood information of field nodes to obtain structural feature vectors. .

[0035] in, This represents the encoding operation of a graph neural network structure, through aggregation and fields. The association information of adjacent nodes yields a structural feature vector containing the table to which the field belongs, primary and foreign key relationships, and information about associated fields.

[0036] Based on the above three types of features, the fields are obtained. The field represents : , in, , and These are the trainable mapping matrices corresponding to semantic features, statistical features, and structural features, respectively, used to map the three types of features to a unified representation space. This is a feature splicing operation. To normalize. This yields the set of field representations. .

[0037] S1-2 Field Type and Visual Role Prior Determination The field type is determined based on the database declaration type and the value format and data format of the actual data records. Fields are categorized into numeric, categorical, time, boolean, and text types. The type is represented as The types of all fields constitute the field type set. .

[0038] The system establishes visual role priors based on field types and their statistical characteristics. For fields... Its visual role prior representation is: .

[0039] in, This describes the operation for determining candidate visual channels based on field type and statistical characteristics.

[0040] Specifically, time-type fields are given priority as candidates for the horizontal axis; numerical fields are given priority as candidates for the vertical axis; low-cardinality category fields are given priority as candidates for color, series, or legend; and high-cardinality category or text fields are given priority as candidates for labels or tooltips.

[0041] The visual role priors of all fields constitute the visual role prior set: .

[0042] S1-3 Standardization of Chart Templates, AST Analysis and Constraint Chart Construction For chart template library templates in The system reads the chart configuration code from the template, standardizes the data field configuration format and configuration item names, and removes redundant style configurations that do not affect the data mapping relationship, thus obtaining a standardized template. All standardized templates constitute a standardized template library. .

[0043] Standardized templates are configured according to the syntax rules of the template configuration language. The code is parsed and converted into an Abstract Syntax Tree (AST), and nodes that can be bound to database fields are located based on predefined configuration item names. Visual channels such as the horizontal axis, vertical axis, series, color, legend, label, and tooltip are extracted from the AST. For templates... Its visual channel set is represented as . This represents the visual channel representation of the i-th database-bindable field in the template.

[0044] Simultaneously, visual channel constraints are established based on the actual data mapping relationships in the template. When the template's vertical axis configuration item requires a numeric field, type constraints are established between the vertical axis and the numeric field; when the template sets both series and color channels, compatibility constraints are established between series, color, and category fields. These constraints are further organized to form a unified set of visual channel constraints for the template: template The visual channel constraint is represented as ; in, template Constraints between different visual channels and between visual channels and field types, number of fields, and data organization methods. This includes: the vertical axis can only be bound to numeric fields, and series channels and color channels should be bound to category fields.

[0045] Finally, we obtain the visual channel representation set of all templates. and template constraint set .

[0046] S1-4 Cross-modal contrastive pre-training By utilizing historically generated successful chart records, the fields actually used and their corresponding visual channels are used as positive samples. Simultaneously, based on the set of field types... and template constraint set Negative samples were constructed by replacing fields of the same type but different semantics and by replacing fields that violated template channel constraints with visual channels.

[0047] Specifically: Field representation based on S1-1 Visual channel representation obtained from S1-3 Calculated fields With visual channel similarity between : , in, and These represent the mapping matrices for field representation and visual channel representation, respectively. This indicates the semantic similarity between the field and the visual channel.

[0048] Further optimization of field-visual channel similarity is achieved by sequentially using positive and negative samples. This ensures that successfully bound field-channel combinations achieve high similarity, while combinations of the same type but semantically mismatched or violated rules... Combinations with mid-channel constraints achieve lower similarity. The contrastive learning objective can be expressed as: , in, This indicates the number of positive samples during training; Indicates the first The similarity of the correct field – visual channel combination; Indicates the relationship with the first One incorrect field – similarity of visual channel combinations; This indicates the number of negative samples corresponding to each positive sample; This represents the temperature parameter.

[0049] After training, the system obtains the field mapping matrix. Visual channel mapping matrix The system represents fields as collections. Field type collection Visual character prior set Visual channel representation set Template constraint set and training and The data is stored uniformly according to field information, template information, and field-visual channel association information to form a multi-source chart knowledge base. .

[0050] As a refinement of the above embodiments, step S2 is to execute a counterfactual dual-path retrieval enhancement mechanism, which includes: This step involves building a multi-source graph knowledge base based on user natural language queries Q and S1. The input is Q. Q is obtained by the system receiving natural language chart requests from the user, such as "statistics on sales trends over the past three years by month".

[0051] The network structure diagram of the graph information generation model based on retrieval enhancement and natural language processing is as follows: Figure 2 As shown, the specific steps include: S2-1 Query Intent Map Analysis Parse the user's natural language query Q into a query intent graph. The nodes in the query intent graph include metrics, grouping dimensions, filtering conditions, aggregation methods, time granularity, and chart structure preferences, while edges represent the modifications or dependencies between intent elements.

[0052] For example, the "Monthly Sales Trend Statistics" should identify "Sales" as the indicator, "Month" as the time group, and "Trend" as the chart structure preference. Among them, "Month" and "Sales" are connected through the grouping relationship, and "Trend" and "Sales" are connected through the chart structure preference relationship, thus indicating that the time grouping affects the sales statistics results, and the trend chart is used for display.

[0053] Query intent graph The query intent representation is obtained after encoding by a language model and propagation through a graph neural network. ,in, This indicates the query intent encoding operation. This represents a trainable query intent graph neural network.

[0054] The trainable query intent graph neural network consists of a node input mapping layer, two graph attention network layers, and a global readout layer. The node input mapping layer maps the linguistic encoding features of each node in the query intent graph to a unified dimension. The text content corresponding to each node is encoded by the pre-trained language model BERT to obtain the initial node feature matrix. Subsequently, neighborhood information is propagated through a two-layer graph attention network, resulting in... and The first-layer graph attention network aggregates the first-order neighbor information of each node. The second-layer graph attention network further aggregates the second-order neighbor information based on the output of the first layer, thereby learning the dependencies between metrics, grouping dimensions, filtering conditions, time granularity, and graph structure preferences. Subsequently, the global readout layer aggregates the representations of all nodes to obtain a global representation of the query intent. .

[0055] S2-2 Generation of Data Query Candidates and Template Candidates (1) Data-side candidate generation. The system generates candidates based on the query intent. The set of fields obtained by combining S1 The database fields related to the query intent are retrieved from the field type set T and the visual role prior set P.

[0056] Specifically, the system calculates the query intent representation. Fields represent collections Each field in the table represents the semantic similarity between the elements, and the elements are sorted from highest to lowest similarity, retaining the top-ranked elements. indivual, The relevant fields are used as candidate fields. Subsequently, multiple data query candidates are generated based on these candidate fields. , The b-th data query candidate includes the SQL statement, the database fields used, and the corresponding aggregation, filtering, and grouping information. The maximum candidate range for b is 10.

[0057] (2) Template-side candidate generation. The system generates candidates based on the query intent. The visual channel representation set V, the visual role prior set P, and the template constraint set obtained by combining S1 Visual channel mapping matrix Standardized template library established from S1 Multiple template candidates were recalled in China. .

[0058] Specifically, the system calculates the query intent representation. The similarity between the visual channel representations of each chart template and the template constraint set is calculated. Constraint filtering is applied to candidate templates, and the top-ranked templates are retained after being sorted by their overall matching score. There are 1 template as template candidates, and k is 5 in this invention. This represents the c-th icon template for the recall.

[0059] Subsequently, the data query candidates With template candidates Combine them to form a SQL-template joint candidate set. For cases where the user explicitly specifies a chart type that conflicts with the format of the queried data, the user-specified template is retained as a candidate, and then... Determine potential constraint conflicts and allow other compatible templates to enter the candidate set.

[0060] S2-3 Joint Utility Calculation For each SQL-template joint candidate The system indicates based on the query intent SQL candidates And the template visual channel representation V, calculate the initial joint utility: , in, express The initial joint utility; This indicates the semantic matching utility between query intent and SQL candidates and chart templates; This indicates the structural adaptation effect between the fields involved in the SQL and the template visual channel; This represents the corresponding effect weight. It is obtained by calculating cosine similarity, that is: .

[0061] The structural adaptation utility is calculated using template constraint rules. This is based on the field type set obtained from S1. Visual channel representation set and template constraint set Check each field in the SQL candidate to see if it meets the visual channel requirements of the template: A value of 1 is recorded when the field type matches the visual channel requirements, and 0 otherwise; a value of 1 is recorded when the number of fields meets the template requirements, and 0 otherwise; a value of 1 is recorded when the field organization conforms to the template constraints, and 0 otherwise. Based on these results, the total field type matching result is obtained. Total number of fields matching results Overall data organization method matching results The structural fit utility is obtained by weighted summation of the three results. .

[0062] S2-4 Lightweight Execution and Template Sketch Verification For the SQL-template combined candidate set For each candidate in the list, the corresponding SQL is executed in a read-only, time-limited, and quantity-limited controlled database environment, and the query results are obtained: .

[0063] in, This represents the sub-table of the query results obtained from executing the b-th SQL query candidate. This indicates the SQL execution operation.

[0064] The union candidate is marked as non-executable if one of the following conditions is met: (1) SQL execution failed; (2) The query result is empty; (3) The SQL reference field does not exist; (4) The field type is incompatible with the template requirements; (5) The size of the query results exceeds the preset security threshold; (6) Involves unauthorized data fields.

[0065] For candidates executed via SQL, the system will process the query results in a sub-table based on the visual channel configuration defined in the template. The corresponding fields are populated into the horizontal axis, vertical axis, series, and color configuration items to generate a lightweight chart sketch without complex styles and custom scripts.

[0066] Sketch verification uses template constraint sets. The rule-based validation process is completed without manual intervention. The system checks the following in sequence: 1) Whether the required visual channels are satisfied; 2) Whether the field types are satisfied. The constraints in the template are as follows: 3) Whether the data is not empty; 4) Whether the cardinality of the category field is within the range that the template can handle; 5) Whether the horizontal axis, vertical axis, and series configurations can form a valid chart structure. The draft validation is considered successful only if all of the above rules are met. For joint candidates that pass both execution and draft validation, the results are used to determine the validity of the chart. Calculate execution utility And update the joint utility: , , in, This indicates the SQL execution status. It is set to 1 when the SQL executes successfully in a controlled database environment without any permission, timeout, or field errors; otherwise, it is set to 0. This indicates the data integrity of the query results. It is set to 1 if the data is complete and meets the field type requirements, and 0 otherwise. This indicates whether the sketch meets the diagram structure requirements. It is set to 1 if it meets the visual channel and structural constraints specified in the template, and 0 otherwise. , and These are the corresponding weights. Wherein, This indicates the combined utility after execution and verification. To apply utility weights. The final output consists of validated joint candidates and their corresponding query results. and combined effects Candidates that fail the verification process are eliminated.

[0067] S2-5 Counterfactual Field Replacement Validation For the SQL-template joint candidate set verified by S2-4 The system uses the fields obtained in S1 to represent the set based on the metrics, grouping dimensions, and time field in the query. and field type collection Select fields with the same data type as the field to be tested, but with different semantics, as replacement candidates.

[0068] For example, replace the indicator field "sales revenue" with "number of employees," which is also a numeric type but has a different semantic meaning, or replace the time field "order month" with "shipment month."

[0069] For the original joint candidates, the joint utility is After replacing the key fields, regenerating and executing the corresponding SQL yields the counterfactual joint utility. .

[0070] Furthermore, calculate field replacement sensitivity. ; in, Indicates the degree of impact of field replacement on joint utility; when The impact of field replacement is relatively weak. It has a certain influence. Fields with high importance are considered. Counterfactual replacements are performed on the metric, grouping, and time fields in the query, and the importance of the fields is determined based on the change in combined utility before and after the replacement.

[0071] S2-6 Joint Candidate Ranking and Output The system obtains the joint utility based on S2-4 and the field replacement sensitivity obtained from S2-5 A comprehensive ranking is performed on the joint candidates that have passed both execution and sketch validation: , in, Indicates joint candidates The final score; This indicates the weight of the field replacement sensitivity, with a value of 0.1.

[0072] The system follows Sort the candidates from highest to lowest score to obtain the joint candidates with the highest scores: .

[0073] in, This indicates the SQL query that is ultimately retained. This represents the final chart template. The corresponding final query result is: ,in, This represents the final data sub-table that has undergone verification.

[0074] Ultimately, a joint search result set is formed. . This represents the joint utility change resulting from counterfactual replacement of the key field corresponding to the highest-scoring joint candidate, used to characterize the importance of the key fields in the final candidate to the query-data-template matching relationship.

[0075] As a refinement of the above embodiment, step S3 is a field-visual channel alignment network for optimal transmission of neural symbols, which includes: S3-1 Neural Symbol Matching Cost Construction Based on the final data sub-table in the joint search result set and final chart template The field representation set obtained from S1 and visual channel representation set Extract the final chart template. Corresponding visual channel set and template constraint set And determine the set of fields to be aligned. Let the soft binding probability of a field assigned to the visual channel be... , which is the probability of assigning a field to the visual channel during the current field-visual channel alignment process, and its value ranges from [0,1].

[0076] The neural matching cost between the computation field and the visual channel ; in, The multilayer perceptron matching network performs feature fusion on the two vectors mentioned above through a fully connected layer, a non-linear activation function, and an output layer, and outputs a matching score between the field and the visual channel; the higher the matching score, the more suitable the field and the visual channel are to be bound.

[0077] Furthermore, the matching score is negativeized to convert it into the matching cost in the optimal transmission. The smaller the cost, the higher the degree of matching.

[0078] Subsequently, a symbolic penalty term is constructed: , in, Indicates field type compatibility judgment: 0 is taken when the set of field types meets the field types required by the visual channel, otherwise 1 is taken; The category cardinality compatibility check is as follows: 0 is used when the field cardinality is within the range allowed by the visual channel, and 1 is used otherwise. Template constraint judgment: When the field and the visual channel satisfy the template constraint set. The value is 0 if the corresponding field source, syntax, and configuration constraints are specified, otherwise it is 1. , and These represent the penalty weights corresponding to the field type, class cardinality, and template constraint, respectively, all of which are non-negative parameters that can be trained or determined through a validation set.

[0079] The neural matching cost is added to the symbolic penalty term to obtain the field-visual channel matching cost; for the field replacement sensitivity in S2 The cost of further leveraging the importance of the top 10% of key fields for visual channel matching is: ; in, The weights are adjusted based on sensitivity, and the final result is the comprehensive matching cost matrix. .

[0080] S3-2 Field Capacity and Channel Requirements Constraints Based on template constraint set In the final icon template The corresponding constraint information determines the field requirements for each visual channel and the allowed number of times each field can be reused.

[0081] The system creates a capacity vector for the field. ; Representation field The maximum number of visual channels allowed to be bound, for fields that are allowed to be reused. Fields that are not allowed to be reused satisfy the following conditions: .

[0082] At the same time, a demand vector is established for the visual channel. ; Indicates visual channel The number of fields to be bound; the required visual channel is 1, and the optional visual channel can be satisfied by an empty field node.

[0083] S3-3 Channel Dependency Graph and Structural Penalty According to the template constraint set The visual channel combination relationships recorded in the data are used to establish a channel dependency graph.

[0084] For example, line charts require the horizontal and vertical axes to form a combination of "time or ordered category - value"; pie charts require both category and value channels to exist simultaneously; and when series channels and color channels are bound to fields, their category cardinality needs to remain compatible.

[0085] set up For a field-visual channel soft-binding matrix, the chart structure penalty is defined as follows: , in, This represents the visual channel combination relationship within the template constraint set; Represents the constraint weights of the corresponding channel relationships; This function represents the compatibility between the binding states of two visual channels, used to calculate the compatibility degree between the current binding states of the two visual channels based on template constraints. For channel combinations that can be directly determined by template rules, if the constraints are satisfied... Violation of the constraints .

[0086] S3-4 Entropy Canonical Optimal Transport Solution An optimal transport model with entropy regularization and structural penalty is adopted to perform global soft alignment between fields and visual channels. The optimization objective is: , in, This represents the cost of matching each field – the visual channel. Corresponding soft binding probability The total cost of field-visual channel matching is obtained by performing element-wise multiplication and summation; the second term represents the entropy regularization term, used to maintain the stability of the solution; the third term represents the calculation of the current soft binding matrix. The degree of violation of the visual channel combination constraint is calculated as follows: This is used to penalize soft binding schemes that do not satisfy the chart structure combination relationship; Represents the entropy regularity coefficient; This represents the structural penalty weight. The system uses Sinkhorn iteration to solve the above optimization problem, obtaining the optimal soft binding matrix for the field-visual channel: .

[0087] S3-5 Discrete Projection and Legality Verification Since each element in the optimal soft-binding matrix represents a binding probability, further... The projection is a discrete binding result that can be directly used for chart code generation. Specifically, with The binding probability of each field-channel is used as the candidate matching weight. The field-visual channel combination with the highest total binding probability is selected, and combinations that violate template constraints are eliminated during the selection process.

[0088] After obtaining the candidate discrete binding results, the system uses the final icon template. Corresponding template constraint set The process involves sequentially validating field capacity, required visual channels, field type, category cardinality, and visual channel combination relationships. If any constraints are violated, the corresponding binding is canceled, and a new candidate is selected from the remaining candidates until a discrete binding result that satisfies all constraints is obtained. The final result is a discrete field-visual channel binding matrix. .

[0089] in, Representation field Ultimately bound to the visual channel Otherwise, take 0.

[0090] S3-6 Ambiguity Identification and Candidate Retention For each visual channel The binding confidence is calculated based on the optimal soft binding matrix: ; in, Indicates visual channel The highest binding probability. When If the binding probability is lower than a preset confidence threshold, or the difference between the highest and second-highest binding probabilities is lower than a preset interval threshold, the system determines that the visual channel has binding ambiguity. For ambiguous visual channels, a limited number of candidate fields that satisfy the template constraints are retained, and the replacement sensitivity of the corresponding fields is considered. Establish a candidate set: .in, Indicates visual channel A set of low-confidence candidate fields.

[0091] Finally, a unified set of field-visual channel aligned results is formed. .

[0092] As a refinement of the above embodiments, step S4 is an AST constraint decoding and sandbox execution automatic error correction method, which includes: S4-1 Generating Context Construction according to and Construct a code generation context.

[0093] The generation context includes: user query Q, and the final data sub-table. The actual field names and field types, and the final chart template. Discrete field-visual channel binding matrix Low-confidence candidate field set Template constraints and target chart library version are the input information for generation. Only [the following information is provided]. The actual fields in the data are used as candidates for generation, and adjustments to the candidate fields are only allowed from... This avoids generating fields that do not exist in the database.

[0094] The code generation context is represented as . For context encoding operations, the code generation context representation is formed by encoding and concatenating the field names and types, chart template AST, field-visual channel binding relationships, and low-confidence candidate fields in the data sub-table. .

[0095] S4-2 Code Generation Based on AST Constraints With the final chart template Using an Abstract Syntax Tree (AST) as the skeleton, configurations such as data sources, horizontal axes, vertical axes, series, colors, legends, and tooltips are generated node by node along the AST path, instead of regenerating the entire code. Let the first... The decoding status of each AST node is as follows: ,in, This represents an AST-constrained decoder; Indicates the current AST node status; This represents the original decoded output of the current node.

[0096] In the t-th decoding step, the system constructs a joint mask based on the final chart template AST, field binding results, and field type information. And determine the constraint decoding probability as: ;in, This represents the configuration item generated by the t-th AST node. After all AST nodes have been decoded, the initial graph code is obtained. ; in, This indicates the code for reconstructing the complete chart configuration based on the results generated by each node.

[0097] S4-3 Sandbox Execution Will Inject into an isolated sandbox for execution. The sandbox restricts network access, CPU, memory, and execution time, loads only the target chart library version, and obtains the sandbox execution results. ; in, It indicates the results of sandbox execution, including execution status, error logs, rendering results, and chart quality information. To isolate the sandbox execution mechanism.

[0098] The overall reward is calculated based on the sandbox performance results: , in, Indicates the correctness of the code syntax; Indicates whether the chart was successfully rendered; This indicates the chart layout and readability; This indicates the degree of consistency between the chart and the user's query intent; , , and These represent the corresponding weights. The code execution success status is determined based on the process exit status, exception information, and execution logs returned by the sandbox; code execution is considered successful if no syntax errors, runtime exceptions, or timeouts occur during execution, and the sandbox process terminates normally. The chart rendering success status is determined based on whether the chart object was successfully created, whether the rendering interface returned normally, and whether a valid image was generated. The overall reward R is used to evaluate the overall quality of the current chart code from four aspects: whether it can execute normally, whether it can render successfully, whether the chart is reasonable, and whether it meets the user's intent.

[0099] If the execution is successful, proceed to S4-5 to output the result; otherwise, generate a structured error message and input S4-4.

[0100] S4-4 Structured Error Diagnosis The results of the sandbox execution The execution failure logs are converted into structured error messages. ; in, Indicates the error category; Indicates the path of the corresponding AST node; Indicates the fields involved; Indicates from Replaceable candidate fields. Error messages are prioritized according to syntax errors, field errors, type errors, and layout errors, and serve as input for S4-5 minimum patch repair. The structured error messages also serve as feedback for subsequent model strategy optimization.

[0101] Minimal patch fixes for S4-5 reinforcement learning driven learning The code repair is modeled as a Markov decision process, where the state includes the current AST node, error message, field bindings, and historical repair trajectory; actions include replacing candidate fields, adding required fields, and deleting invalid nodes. The policy network uses a near-end policy to optimize and update the repair strategy; after each patch repair is completed, the [number]th [item] is generated. Wheel repair code . This indicates a patch repair operation; This indicates the current round of repair actions.

[0102] Return to S4-3 for static analysis and sandbox execution. Stop the fix when the chart code executes successfully and the overall reward reaches the quality threshold, and record the final code as... . This indicates the final executable chart code. The final chart generation result is obtained. ; in, This represents the repair log, which records the AST nodes, repair actions, reward changes, and termination reasons for each round of repair, and is used for result reproduction and subsequent model optimization.

[0103] The final chart generation result is used to transform the results after data querying, field-visual channel alignment and execution verification into a chart that can be run directly, and serves as the final visualization output of this invention.

[0104] Simulation results (one) Figure 3The Top-1 matching accuracy of different matching strategies across six visual channels was compared. Overall, the proposed method achieved the highest accuracy across all visual channels, reaching 96.4%, 97.8%, and 94.9% for the X-axis, Y-axis, and color channels, respectively, and 93.7%, 95.3%, and 92.8% for the series, legend, and tooltip channels, respectively, with an average accuracy of 95.15%. Compared to methods using only semantic similarity, the proposed method improves accuracy by approximately 7.5 percentage points on average, indicating that introducing neural-symbol matching, field type, and template constraints effectively reduces the problem of semantically similar fields but mismatched visual roles. Especially in more complex visual channels such as series and tooltip, the proposed method maintains high accuracy, demonstrating that global field-visual channel alignment can improve matching stability under complex chart structures.

[0105] (two) Figure 4 This is a sales trend chart automatically generated based on natural language commands. The result demonstrates the complete generation process of the proposed method, from natural language query to final visualization. Users only need to input the natural language requirement of "statistically analyze the sales trend of each month over the past three years." The system automatically identifies time groupings and sales indicators, completes relevant field retrieval, chart template matching, and field-visual channel alignment, and generates the corresponding time-series line chart. The generated result accurately expresses the sales trend over 36 months while retaining key data features such as annual growth and peak values, demonstrating that the proposed method can effectively convert natural language chart requests into structurally correct, data-consistent, and directly executable visualizations.

[0106] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for generating chart information based on retrieval enhancement and natural language processing, characterized in that, Includes the following steps: Obtain data field information and chart template library from the database; perform feature extraction, field type identification, and prior modeling of visualization roles on the database fields to obtain field representations; parse the chart templates to extract the visual channels in the templates and establish a set of template constraints; A multi-source graph knowledge base is constructed through cross-modal contrastive pre-training based on field representation, visual channel representation, and template constraint set; Parse user natural language queries into chart intents; Data query candidates and chart template candidates are generated based on a multi-source chart knowledge base and then combined into a joint candidate. The joint candidate is then validated to verify its chart execution feasibility. Counterfactual substitutions are performed on the fields involved in the joint candidate, and the joint candidate is filtered based on the changes in joint utility before and after the substitution to obtain a joint retrieval result set. Based on the joint retrieval result set, the matching cost between the field and the visual channel is calculated through a neural network, and a comprehensive matching cost matrix is ​​obtained by combining a constraint-based symbolic penalty term. The optimal transmission method is used to solve the alignment result between the field and the visual channel, forming a field-visual channel alignment result set. Based on the joint retrieval result set and the field-visual channel aligned result set, a code generation context representation is constructed, and the abstract syntax tree of the chart template is used as the generation skeleton to perform constraint decoding and generate chart code; Execute the chart code and calculate the overall reward based on the execution result; If the execution fails, the chart code will be repaired based on the error message, and the check and execution will be repeated until the termination condition is met, and the chart generation result will be output.

2. The chart information generation method based on retrieval enhancement and natural language processing according to claim 1, characterized in that, The feature extraction includes semantic feature extraction, statistical feature extraction, and structural feature extraction, specifically including: Read the table name, field name, field comments, and field information of the same table, and perform joint encoding through a text semantic encoder based on a pre-trained language model to obtain semantic features; Statistical analysis is performed on the actual data records of the fields to obtain statistical characteristics. Among them, the statistical characteristics of numerical fields include the range of values, mean, standard deviation, quantiles and missing rate; the statistical characteristics of categorical fields include the number of unique values, category entropy and the proportion of high-frequency categories; and the statistical characteristics of time fields include the time span and time granularity. Based on the table, field, primary key, and foreign key information in the database schema, a database schema graph is constructed using fields as graph nodes and primary key associations and foreign key references between fields as graph edges. Using the database schema graph as input, a graph neural network is used to aggregate neighborhood information of field nodes to obtain structural features. After mapping semantic features, statistical features, and structural features to a unified representation space through their respective trainable mapping matrices, they are concatenated and normalized to obtain the field representation. The field type identification and visualization of role prior modeling specifically includes: Based on the database declaration type and combined with the value form and data format of the actual data records in the field, the fields are divided into numeric, categorical, time, boolean, and text types, forming a set of field types; Visual role priors are established based on field types and their statistical characteristics. Time-type fields are given priority as candidates for the horizontal axis; numerical fields are given priority as candidates for the vertical axis; low-cardinality categorical fields are given priority as candidates for color, series, or legend; and high-cardinality categorical or text fields are given priority as candidates for labels or tooltips. The visual role priors of all fields constitute the visual role prior set. The chart template is parsed to extract the visual channels and a set of template constraints is established, specifically including: Read the chart configuration code of the templates in the chart template library, unify the data field configuration format and configuration item name of the templates, and delete redundant style configurations to obtain standardized templates, thus forming a standardized template library; The standardized template is parsed into an abstract syntax tree according to the syntax rules of the template configuration language. The nodes that can be bound to database fields are located according to the predefined configuration item names. The visual channels of the horizontal axis, vertical axis, series, color, legend, label and tooltip are extracted from the abstract syntax tree to obtain the visual channel representation set of the template. Visual channel constraints are established based on the actual data mapping relationships in the template, forming a set of visual channel constraints for the template. The visual channel constraints include the constraints between each visual channel and between the visual channel and the field type, number of fields, and data organization method.

3. The chart information generation method based on retrieval enhancement and natural language processing according to claim 2, characterized in that, The construction of the multi-source chart knowledge base specifically includes: By utilizing historically generated chart records, the fields actually used and their corresponding visual channels are used as positive samples, and negative samples are constructed by replacing fields of the same type but different semantics and replacing fields that violate template channel constraints with visual channels. The semantic similarity between the field representation and the visual channel representation is calculated, and the positive and negative samples are used for comparative learning to make the successfully bound field-channel combination obtain high similarity, and the field-channel combination that does not match semantically or violates the channel constraints obtain low similarity, thereby training to obtain the field mapping matrix and the visual channel mapping matrix. The field representations, visual channel representations, template constraint sets, and the trained field mapping matrices and visual channel mapping matrices are stored uniformly to form the multi-source chart knowledge base.

4. The chart information generation method based on retrieval enhancement and natural language processing according to claim 1, characterized in that, The process of parsing user natural language queries into chart intents specifically includes: The user's natural language query is parsed into a query intent graph. The nodes in the query intent graph include indicators, grouping dimensions, filtering conditions, aggregation methods, time granularity, and chart structure preferences. The edges represent the modification or dependency relationships between the intent elements. The query intent graph is encoded by a language model and propagated through a graph neural network to obtain a query intent representation. The graph neural network consists of a node input mapping layer, two graph attention network layers, and a global readout layer. The node input mapping layer maps the language encoding features of each node to a unified dimension. The first graph attention network aggregates the first-order neighbor information of each node. The second graph attention network aggregates the second-order neighbor information based on the output of the first layer. The global readout layer aggregates the representations of all nodes to obtain a global representation of the query intent.

5. The chart information generation method based on retrieval enhancement and natural language processing according to claim 4, characterized in that, The generation of data query candidates and chart template candidates based on a multi-source chart knowledge base specifically includes: Calculate the semantic similarity between the query intent representation and each field representation in the field representation set, sort them from high to low similarity and retain the top n fields as candidate fields, generate b data query candidates based on the candidate fields, and each data query candidate includes the SQL statement, the database fields used and the corresponding aggregation, filtering and grouping information; Calculate the matching similarity between the query intent representation and the visual channel representation corresponding to each chart template, and perform constraint filtering on the candidate templates in combination with the template constraint set. After sorting according to the comprehensive matching score, retain the top k templates as template candidates. The data query candidates and template candidates are combined to form a SQL-template joint candidate set. In cases where the user explicitly specifies a chart type but it conflicts with the form of the query data, the user-specified template is retained as a candidate and its potential constraint conflicts are judged, while other compatible templates are allowed to enter the candidate set.

6. The chart information generation method based on retrieval enhancement and natural language processing according to claim 5, characterized in that, The specific steps for performing verification on the joint candidate include: For each SQL-template union candidate, execute the corresponding SQL in a read-only, time-limited, and quantity-limited controlled database environment to obtain a query result sub-table. The union candidate is marked as unexecutable when any of the following conditions occur: SQL execution fails; query result is empty; SQL reference field does not exist; field type is incompatible with template requirements; query result size exceeds a preset security threshold; or it involves unauthorized data fields. For joint candidates executed via SQL, based on the visual channel configuration defined in the template, the corresponding fields in the query result sub-table are filled into the horizontal axis, vertical axis, series, and color configuration items to generate a lightweight chart sketch without complex styles and custom scripts. The sketch is then checked sequentially based on the template constraint set: whether the required visual channels are satisfied, whether the field types meet the constraints, whether the data is not null, whether the cardinality of the category field is within the range that the template can handle, and whether the horizontal axis, vertical axis, and series configurations can form a valid chart structure. The sketch is considered to have passed the verification only if all of the above checks are satisfied. For joint candidates that pass both execution and sketch validation, the execution utility is calculated based on the SQL execution status, the data integrity of the query results, and the structural compliance of the sketch. The joint utility is then updated based on the execution utility. Joint candidates that fail validation are eliminated. The formula for updating the joint utility is as follows: , ， ， in, For the purpose of execution; This indicates the SQL execution status. It is set to 1 when the SQL executes successfully in a controlled database environment without any permission, timeout, or field errors; otherwise, it is set to 0. This indicates the data integrity of the query results. It is set to 1 if the data is complete and meets the field type requirements, and 0 otherwise. This indicates whether the sketch meets the diagram structure requirements. It is set to 1 if it meets the visual channel and structural constraints specified in the template, and 0 otherwise. , and For the corresponding weights; This indicates the combined utility after execution and verification. For initial joint utility; Utility weights for execution; Indicates SQL-template combined candidate The initial joint utility; The semantic matching utility between query intent and SQL candidates and chart templates is represented by cosine similarity calculation. This indicates the structural adaptation effect between the fields involved in the SQL and the template visual channel; Indicates the corresponding effect weight; This indicates the query intent; SQL candidate; For template visual channel representation; This represents the c-th icon template for the recall. This is the b-th data query candidate; A collection of field types; For template constraint set; This represents the sub-table of the query results obtained by executing the b-th SQL query candidate; The calculation method is as follows: Based on the set of field types, the set of visual channel representations, and the set of template constraints, check one by one whether the field type, field quantity, and field organization method in the SQL candidate meet the visual channel requirements corresponding to the template. If they meet the requirements, record 1; otherwise, record 0. The total field type matching result, the total field quantity matching result, and the total data organization method matching result are weighted and summed to obtain the structure adaptation utility.

7. The chart information generation method based on retrieval enhancement and natural language processing according to claim 6, characterized in that, The process involves performing counterfactual substitution on the fields involved in the joint candidates, filtering the joint candidates based on the changes in joint utility before and after the substitution, and obtaining a joint retrieval result set; specifically including: For the validated SQL-template joint candidate set, select fields with the same type as the field to be tested but different semantics as replacement candidates. Perform counterfactual replacements on the indicator, grouping dimension and time fields in the query, regenerate and execute the corresponding SQL to obtain the counterfactual joint utility. Calculate the field replacement sensitivity, which is the change in combined utility before and after the field replacement; The validated joint candidates are weighted and sorted based on the updated joint utility and field replacement sensitivity. The joint candidate with the highest score, its corresponding final SQL query, final chart template, final data sub-table, and the joint utility changes resulting from the counterfactual replacement of the key fields corresponding to the joint candidate with the highest score are obtained to form a joint retrieval result set.

8. The method for generating chart information based on retrieval enhancement and natural language processing according to claim 7, characterized in that, The comprehensive matching cost matrix is ​​obtained as follows: A multilayer perceptron matching network is used to fuse the features of the field representation and the visual channel representation, outputting the matching score between the field and the visual channel, and the negative value of the matching score is converted into a neural matching cost. A symbolic penalty term is constructed, which includes: a field type compatibility penalty, which is 0 when the field type meets the field type requirements of the visual channel, and 1 otherwise; a category cardinality compatibility penalty, which is 0 when the field cardinality is within the range allowed by the visual channel, and 1 otherwise; and a template constraint penalty, which is 0 when the field and the visual channel meet the corresponding field source, syntax, and configuration constraints in the template constraint set, and 1 otherwise; each penalty is multiplied by its corresponding non-negative penalty weight. The neural matching cost is added to the symbolic penalty term to obtain the field-visual channel matching cost. The field-visual channel matching costs of the database fields are combined to obtain the comprehensive matching cost matrix. For critical fields whose sensitivity to field replacement exceeds a preset threshold, the cost of field-visual channel matching is... as follows: ， in, For the cost of neural matching, For symbolic penalty terms, Adjust the weights for sensitivity. The joint utility change resulting from counterfactual replacement of the key field corresponding to the joint candidate with the highest score.

9. The method for generating chart information based on retrieval enhancement and natural language processing according to claim 8, characterized in that, The alignment results between the field and the visual channel are obtained by using optimal transmission, forming a field-visual channel alignment result set; the specific method is as follows: Based on the constraint information in the template constraint set corresponding to the final chart template, determine the field requirement vector for each visual channel. and the capacity vector of each field The capacity vector represents the maximum number of visual channels that a field can be bound to. In the field requirement vector, the required visual channel is 1, and the optional visual channel can be satisfied by an empty field node. The optimization objective is constructed by weighting the total cost of field-visual channel matching, the entropy regularization term, and the structure penalty term. The optimization objective is as follows: ， ， in, This represents the cost of matching each field-visual channel. Corresponding soft binding probability The total cost of field-visual channel matching is obtained by performing element-wise multiplication and summing. Represents the entropy regularization term; Indicates the structural penalty term; Represents the entropy regularity coefficient; Indicates the structural penalty weight; For field-visual channel soft binding matrix; This is the set of template constraints corresponding to the final chart template; This represents the visual channel combination relationship within the template constraint set; Represents the constraint weights of the corresponding channel relationships; A compatibility function representing the binding states between two visual channels; Indicates the relationship with the first The binding status of another visual channel that has a template combination constraint relationship with one visual channel. For the prior set of visual characters; The optimal soft binding matrix for the field-visual channel was obtained by solving the optimization objective using the Sinkhorn iteration method. ; Using the binding probabilities of each field in the optimal soft binding matrix as candidate matching weights, the field-visual channel combination with the highest total binding probability is selected, and combinations that violate the template constraints are eliminated during the selection process. Then, the field capacity, required visual channels, field type, category cardinality, and visual channel combination relationship are checked in turn. If there is a constraint violation, the corresponding binding is canceled and reselected from the remaining candidates until a discrete field-visual channel binding matrix that satisfies all constraints is obtained. For each visual channel, the binding confidence is calculated based on the optimal soft binding matrix. When the highest binding probability is lower than the preset confidence threshold, or the difference between the highest binding probability and the second highest binding probability is lower than the preset interval threshold, it is determined that there is binding ambiguity in the visual channel. A preset number of candidate fields that meet the template constraints are retained, and a set of low-confidence candidate fields is established by combining the replacement sensitivity of the corresponding fields. The optimal soft binding matrix, the discrete field-visual channel binding matrix, the binding confidence, and the low-confidence candidate field set are used as the field-visual channel alignment result set.

10. The chart information generation method based on retrieval enhancement and natural language processing according to claim 9, characterized in that, The specific method for generating the chart results is as follows: The code generation context is constructed, including: user query, actual field names and field types in the final data sub-table, final chart template, discrete field-visual channel binding matrix, low-confidence candidate field set, template constraints, and target chart library version information; wherein, only fields that actually exist in the final data sub-table are provided as generation candidates, and adjustments to candidate fields are only allowed from the low-confidence candidate field set; The constraint decoding uses the abstract syntax tree of the final chart template as its skeleton, and generates data source, horizontal axis, vertical axis, series, color, legend and tooltip configuration node by node according to the path of the abstract syntax tree; in each decoding step, a joint mask is constructed based on the abstract syntax tree of the final chart template, field binding results and field type information, and the constraint decoding probability is determined based on the joint mask. After all the abstract syntax tree nodes are decoded, the chart code is reconstructed. The chart code is injected into an isolated sandbox for execution. The isolated sandbox restricts network access, CPU, memory and execution time, and only loads the target chart library version, resulting in a sandbox execution result that includes execution status, error logs, rendering results and chart quality information. If the code executes successfully, a comprehensive reward is calculated based on the sandbox execution results. The comprehensive reward is obtained by weighted summation of the correctness of the code syntax, whether the chart is successfully rendered, the chart layout and readability, and the consistency between the chart and the user's query intent. If the code execution fails or the chart fails to render correctly, the error log returned by the sandbox will be converted into structured error information. The structured error information includes the error category, the corresponding abstract syntax tree node path, the fields involved, and the alternative candidate fields. Based on structured error information, code repair is modeled as a Markov decision process, where the state includes the current abstract syntax tree node, error information, field bindings and historical repair trajectory, and the actions include replacing candidate fields, supplementing required nodes and deleting invalid nodes. The policy network adopts a near-end policy to optimize and update the repair policy. After each patch is completed, static checks and sandbox execution are performed again. The patching stops when the chart code executes successfully and the overall reward reaches the quality threshold. The final executable chart code is output, and a chart generation result is generated that records the final executable chart code, sandbox execution results, discrete field-visual channel binding matrix, and the abstract syntax tree nodes, patch actions, reward changes, and termination reasons for each round of patching.

Images